This is obviously true of action. Whatever views your views on free will, it is indubitable that differing options occur to us, that we compare them, that we prefer some to others, that eventually we elect one and dismiss the rest. More interestingly, James describes the role of selection in perception, and finds it at every level of neural and mental life. The sense organs, to begin with, are insensitive to almost all that happens around them. When they are excited and transmit nervous impulses to the brain, these are sifted for significant patterns (often found on dubious grounds). News of these is relayed to other parts of the brain, which look for more subtle, more detailed, and more broad patterns, until at last we reach our perceptions, grouped together by another process of selection into things. Some of these we attend to; the rest we ignore.
``The mind is at every stage a theatre of simultaneous possibilities. Consciousness consists in the comparison of these with each other, the selection of some, and the suppression of the rest by the reinforcing and inhibiting agency of attention. The highest and most elaborated mental products are filtered from the data chosen by the faculty next beneath, out of the mass offered by the faculty below that, which mass in turn was sifted from a still larger amount of yet simpler material, and so on. The mind, in short, works on the data it receives very much as a sculptor works on his block of stone. In a sense the statue stood there from eternity. But there were a thousand different ones beside it, and the sculptor alone is to thank for having extricated this one from the rest. Just so the world of each of us, how so ever different our several views of it may be, all lay embedded in the primordial chaos of sensations, which gave the mere matter to the thought of all of us indifferently. We may, if we like, by our reasonings unwind things back to that black and jointless continuity of space and moving clouds of swarming atoms which science calls the only real world. But all the while the world we feel and live in will be that which our ancestors and we, by slowly cumulative strokes of choice, have extricated out of this, like sculptors, by simply removing portions of the given stuff. Other sculptors, other statues from the same stone! Other minds, other worlds from the same monotonous and inexpressive chaos! My world is but one in a million alike embedded, alike real to those who may abstract them. How different must be the worlds in the consciousness of ant, cuttlefish, or crab!''James wrote in 1890, and the last century of research into brain and mind have done nothing to diminish our confidence in this (admittedly very general) picture. Indeed, we can now point to parts of the brain which select specific features out of the signals of the sense organs --- cells in the occipital lobe, for instance, which respond only to straight lines at certain angles, or motion, or contrasts of light and darkness --- and we are beginning to understand the more elaborate construction of things out of such elements. From almost any authority on neurobiology, cognitive science, psychology, the philosophy of mind, artificial intelligence or computer science, I could have quoted passages saying substantially the same things as James, though in worse prose and without quite the same emphasis on selection.
I propose, now, to see whether these ideas of selection shed any light on the various uses of ``thinking machines,'' that is to say, computers.
The reader is almost certainly familiar with fractals, whether of the abstract or the naturalistic variety, but is perhaps less likely to know that computer programs have written verse (rhymed, blank and free), short stories, and even a novel. Art critics --- and more particularly, theoretical art critics --- have been understandably interested in these developments. Some have dismissed them as, at best, amusements for the boys in the basement computer lab across campus, a folk art for those whose native language is C. Others --- such as the late O. B. Hardison Jr., whose views are set forth with admirable clarity in Disappearing Through the Skylight --- have been thrown into a kind of ecstasy of obsolescence. ``Gazing at the thirty-nine sea-horses of the Mandelbrot set,'' they say in effect, ``we can see that human art, Art with a capital A, is at an end, not perhaps this week-end, but soon: it is later than you think. The day will come when a human artist could no more rival a computer than than a sprinter out-race a Ferrari. The coming art will be digital, perfect, timeless, inimitable, perhaps incomprehensible. We and all our works shall pass to dust, and only they will remain, dreaming their silicon dreams of unknown space.'' Such, in essence and composite, is the rhetoric. It seems to me to miss the most interesting aspect of the new computer art, which is its human angle, and with it the most plausible future.
The connection between selection and art is, to revert to James, ``obvious''.
``The artist notoriously selects his items, rejecting all tones, colors, shapes, which do not harmonize with each other and with the main purpose of his work. That unity, harmony, `convergence of characters,' as M. Taine calls it, which gives to works of art their superiority over works of nature, is wholly due to elimination. Any natural subject will do, if the artist has wit enough to pounce upon some one feature of it as characteristic, and suppress all merely accidental items which do not harmonize with this.''Now the peculiarity of computer art is that computer programs are very bad at just this sort of pruning. They will make a pattern --- of colors, of sounds, of words --- according to a rule, and that is all. Give, let us say, a fractal program one rule, and it will draw you the corresponding picture; change the rule slightly and it draws another, similar picture. It does not linger over the interesting, balk at the trite, turn away from the boring or disturbing: it is a machine without preferences, ``a gaze blank and pitiless as the sun.''
Artists, notoriously, are different. Even those who use ``found objects'' select the ones they find interesting, relevant or marketable, and eliminate everything else in the world. Selection is inescapable --- or at least not yet escaped. Computer programs, then, are poor artists because their powers of choice are absolutely miniscule; they select not from a pool of possibilities but a handful of drops, often not even that.
Yet the fact remains that computer-made graphics, if not music or literature, are quite popular, even delightful. How can this be?
We have all had the experience of writing out a sentence and then crossing it out, in bits and pieces, putting in new words and phrases until we find ones which fit to our satisfaction; until, that is, we select one sentence out of a mob of candidates. Computer art programs show us the more promising members of this mob. We then pick and choose among them, according to our tastes and purposes. It is a strange man who would put a view of the Mandelbrot set on a condolence-card; and a rash one who would say there is none suitable for this purpose.
In reality, then, the computer is not an independent artist, but a sort of dumb assistant, an automatic producer of first sketches. If the initial attempt is not perfectly satisfying --- and what first sketch is? --- either improve it by hand, or modify the computer's instructions slightly and have it try again. The afore-mentioned novel was written in the first way, most fractal pictures are arrived at in the second.
It may be asked, Must this be so? Must the computer always be just an adjunct, a patient but moronic apprentice? The key, as we have seen, is to give the computer preferences, and this does not sound impossible. Let it produce one picture, one tune, one sonnet, and see whether it satisfy its principles. If not change it - a more drastic change the further the sketch falls short of giving satisfaction - and repeat this cycle until all the criteria are met. In fact, the computer could consider a number of sketches in parallel, working on them simultaneously and combining promising features, and in this way progress much faster than if it had (so to speak) a one-track mind. (Some will recognize this as an application of the technique of ``genetic algorithms.'')
The difficulty, and it is a large one, lies in spelling out those principles in ways simple and clear enough for a computer to act on. It is hard enough to give an intelligible account of why we like a painting, switch off the radio when that tune comes on, gaze at one statue for inspiration and use another for a door-stop. It is easy to despair of ever being able to deduce the Ninth Symphony's superiority to one of its crossed-out early drafts; it may be better not to contemplate even the attempt. Yet without such formal criteria, independent computer art will remain, at most, a curiosity.
I will only close this subject by saying that there has, recently, been some work done in this field, though I do not know if the investigators have considered it in quite this light. In the closing pages of his recent book Strange Attractors, Professor Julien Sprott of the University of Wisconsin describes a survey he and colleagues made of taste in fractal pictures. People were asked to rank different fractal pictures, and these preferences were plotted against the fractal dimension of the picture (a number which measures how rough, convoluted and ``space-filling'' the image is) and the ``Lyapunov exponent'' of the equations which generated it (another number, which measures how quickly the equations amplify small differences in their variables, the fabled ``sensitive dependence on initial conditions.'') Strange as it may seem, people consistently prefer pictures whose fractal dimension and Lyapunov exponents cluster rather tightly about a constant center. If, then, a computer could be programmed to consistently produce images in that area, it would be a small first step towards ``automated taste.'' A fascinating speculation on where this might lead --- uninfluenced, as far as I know, by Prof. Sprott --- is found in Ian McDonald's science fiction novel Scissors Cut Paper Wrap Stone.
It is curious, and I believe not previously noticed, that something very similar is essential to high-energy physics. (Physics also needs normal perception, of course.)
At this point alarms ring in the minds of my colleagues, since we are all too familiar with books on the profound connection between ``the new physics'' and consciousness and various sophomoric distortions of Asian mysticism. The authors of this school are seldom discussed, save by graduate students who laugh at the errors and covet the royalties. Rest assured, I shall not discuss the torture of cats, Buddhist puns, interpretive dance, the Tao of the relativistic Euler-Lagrange equations, the maya-aspects of renormalizable gauge field theories, or even how to find a cheap Chinese restaurant in Copenhagen without a Danish interpreter.
My subject is, instead, rather more massive and solid and sweaty: the detectors attached to particle accelerators. A word or three of reminder about these, too, may not be out of place.
Particle physicists are interested in what the smallest discoverable bits of matter are, and how they behave. They are especially interested in how they behave at very high energies, since these let them probe very short distances and led to unusual (and hence informative) events, like the creation of new kinds of particles. The only practical way to give elementary particles lots of energy is to accelerate them to very high speeds; the electro-magnetic machines which do this are called, imaginatively enough, ``accelerators''. Some accelerators send a beam of particles into a fixed target of more normal matter, say, gold foil. The really high-energy ones collide two beams of particles moving in opposite directions. There are all sorts of fascinating technical issues, on which I may well end up writing a dissertation --- but another time.
More interesting for us than the accelerators are the detectors, the machines which sense what happens when the particles collide. The need for such machines is quite real. The events happen far too quickly (over 10^-23 to, at the most lackadaisical, 10^-10, seconds) and in too small a region (on the order of 10^-18 meters) for human perception.
I come at last to the heart of the matter. Most of the oceans of data from detectors are uninteresting and worthless. Recall that physicists want to learn about unusual, hard-to-achieve or anomalous events; everything else is noise. But common, easily occurring events are by definition the majority; therefore most events are uninteresting. Sturgeon's Law states that ``ninety percent of everything is crap.'' For particle physics, this is wildly optimistic; interesting events can be outnumbered by billions or trillions to one. In theory, combing haystacks for needles is what professors have graduate students for. In practice, not even an army would suffice.
What does suffice is very high speed electronics, working on time-scales of under a microsecond. The lowest level, known as the trigger, scans the signals from the detector for an interesting pattern, usually something very simple, like ``two diametrically opposed detectors activated.'' The data is recorded only if the trigger is (for want of a better word) triggered. Once it is recorded, the computers set to work on it, attempting a more and more detailed reconstruction of the event. At each stage in the reconstruction there are ``cuts'', i.e. some events are selected for their interesting characteristics and the rest discarded. (For instance, we might want events where all the outgoing particles concentrate into two back-to-back jets, and so cut those where lots of other detectors got triggered, along with a diametrically opposed pair.) Great care is lavished on both the design of the cuts and the reconstruction, for figuring out what to ignore is, practically, as important as figuring out what happened. What bubbles up, in the end, are a handful of reconstructions selected --- elected? --- for conscious, human attention.
Piling layers of selection atop each other is essential if we are, reasonably quickly, to direct our resources where we they ought to be most useful; triage is a dramatic example. And in fact successive cuts give experiments remarkable leverage. (See again our back of the envelope calculation.) Physicists have been in love with leverage since Archimedes, but there is a cost, and to illustrate it I shall, with the reader's kind permission, once again quote William James, this time on attention:
``[I]n those puzzles where certain lines in a picture form by their combination an object that has no connection with what the picture ostensibly represents; or indeed in every case where an object is inconspicuous and hard to discern from the background; we may not be able to see it for a long time; but, having once seen it, we can attend to it whenever we like, on account of the mental duplicate of it which our imagination now bears. In the meaningless French words `pas de lieu Rhone que nous,' who can recognize immediately the English `paddle your own canoe?' But who that has once noticed the identity can fail to have it arrest his attention again? When watching for the distant clock to strike, our mind is so filled with its image that at every moment we think we hear the longed-for or dreaded sound. So of an awaited footstep. Every stir in the wood is for the the hunter his game; for the fugitive his pursuers. Every bonnet in the street is momentarily taken by the lover to enshroud the head of his idol. The image in the mind is the attention; the preperception, as Mr. Lewes calls it, is half of the perception of the looked-for thing.In detectors, ``preperception'' takes the form of the hard-wired trigger and programmed cuts. An event which might be fantastically interesting, if only we knew about it, will be sent into oblivion if it does not match our a priori criteria at every step. In this sense, experimenters only find what they are looking for --- if it exists.``It is for this reason that men have no eyes but for those aspects of things which they have already been taught to discern. Any one of us can notice a phenomenon after it has once been pointed out, which not one in ten thousand could ever have discovered for himself. Even in poetry and the arts, some one has to come and tell us what aspects we may single out, and what effects we may admire, before our aesthetic nature can `dilate' to its full extent and never `with the wrong emotion.' In kindergarten instruction one of the exercises is to make the children see how many features they can point out in such an object as a flower or a stuffed bird. They readily name the features they know already, such as leaves, tail, bill, feet. But they may look for hours without distinguishing nostrils, claws, scales, etc., until their attention is called to these details; thereafter, however, they see them every time. In short, the only things which we commonly see are those which we preperceive, and the only things which we preperceive are those which have been labelled for us, and the labels stamped into our minds.''
The analogy between detectors and our view of perception is rather close. (Technological determinists, kindly note that James began writing in 1880, but the first accelerators were built in the 1930s.) It would be rash to claim that large particle detectors are conscious. In them we have perhaps the foundations and basic plumbing (with special attention to sewage disposal) of the building of consciousness; perhaps some scaffolding for the higher floors as well.
Following James, P. N. Johnson-Laird's The Computer and the Mind presents more or less the standard view of the ``artificial intelligentsia and cognitive cognoscenti'' with lucidity and no more detail than a common reader may be expected to accept. Cognitive science is a ``top-down'' approach; good sources for the complentary ``bottom-up'' view of the brain scientists, which is considerably wetter and messier, are William Calvin and George Ojemann, Conversations with Neil's Brain (New York: Addison-Wesley, 1994) and, at a higher level but still very accessible, Shepherd's delightful Neurobiology (Oxford University Press, 1983) and A. R. Luria's The Working Brain (New York: Basic Books, 1973).
More idiosyncratic but still broadly mainstream views can be found in Marvin Minsky, The Society of Mind (artificial intelligence), William Calvin, The Cerebral Symphony (neurology) and Daniel Dennett, Consciousness Explained (philosophy).
The literature on fractals and computer art is swiftly becoming as unsurveyable as that on anything else. James Gleick's Chaos is too well-known to need a plug here. Benoit Mandelbrot's The Fractal Geometry of Nature is recommended only for the strongly mathematical. The picture-books of fractals are beyond counting. In addition to the results of his work on aesthetics, Prof. Sprott's book Strange Attractors contains details of procedures for rapidly making fractal pictures. It is interesting to compare abstract fractals with the pictures in James O'Brien, Design by Accident (New York: Dover, 1964).
Manuel De Landa, War in the Age of Intelligent Machines. New York: Zone Books, 1991 (distributed by the MIT Press). Interesting military history and fascinating, horrifying reports on the latest Pentagon uses of computers and AI, along with very dubious history of philosophy and ideas, smothered throughout in an appalling mis-use of technical terms from dynamics. (Even as metaphors, they don't make much sense.) Alas, I can't find a better book on the subject.
Expand the military section.
Consider whether any finite cognitive entity (ugh! cogitator? cognitator? --- double ugh! --- knower!) wouldn't be forced to be selective, and hierarchically selective at that. (Selection I think is a necessary consequence of finitude; but hierarchies and combinations a la James seem to follow more from needs to accomplish some purposes quickly, i.e. from functions. Cf. Dennett in Elbow Room (especially the discussion of Laplace's Vast and Considerable Intellect) and Simon in Sciences of the Artificial on the nature of ``artifacts.''