The Bactra Review: Occasional and eclectic book reviews by Cosma Shalizi   45


Exploring the Neural Code

by Fred Rieke, David Warland, Rob de Ruyter van Steveninck, and William Bialek

Computational Neurosciences series. MIT Press, 1997

Cells That Go ping, or, The Value of the Three-Bit Spike

Neurons lead a remarkably (someone fond of vile puns might say singularly) punctiform existence. Brief, stereotyped electrical impulses, called action potentials or spikes, enter the neuron at one end through the synapses, and a single spike goes out along the axon at the other (or not, according to circumstances and temperament); after sending a spike a neuron enters what is aptly called its refractory period, when it refuses to spike for any reason whatsoever. That, pretty much, is all: such is the stuff that dreams are made of; sensations, syllogisms, reflexes, rapport, baseball pitches and bowel movements, too. It's been plain for a long time that neurons communicate with these spikes; any doubts were disposed of in the 1920s, when physiologists learned how to record the electrical activity of single neurons while controlling their stimuli: change the stimulus and the pattern of spiking changes reproducibly. Naturally, we suppose the spikes carry information about the stimulus, that they encode it. Naturally, we want to know What is the code? and How is it used? Spikes is the best answer yet, and it starts from thinking carefully about what the organism wants the code for anyway.

It wants the code, of course, so that it can decode signals and see what the rest of the world is like. The task confronting a homunculus residing in the brain, having access only to the incoming trains of spikes, would be to guess --- that is, infer --- what the most likely state of the world is, given that the spike train is thus-and-so. (Shades of Quine's "sensory irritations.") There are no homunculi, but this is, in effect, the problem one part of the brain must solve to make use of information coming from another. If the code is well-designed, this decoding operation will be simple and reliable, and ideally linear. It turns out, after applying Bayes's rule, that lots of neurons --- all the kinds the authors have been able to examine in the necessary detail, or anyhow all those they care to talk about --- can be nearly linearly decoded. More precisely: pass the spike-train through a certain well-prescribed linear filter, the "first Wiener kernel," and you get a good estimate of the mean of the stimuli that will produce that spike-train. Adding quadratic terms to the filter doesn't improve the fit much, so the first Wiener kernel is the neural code for that neuron, or very nearly so, since it tells us how to decipher the spikes.

Now that we can read the code, that we know what the spikes tell us, we'd like to know how much they are telling us, how much information a spike-train, or even a single spike, conveys. So long as we're rummaging around in Norbert Wiener's toolkit for doohickeys like the first kernel, we might as well check if there's anything there which would help with this problem: and of course there is, namely information theory, priority for which goes, however, to Shannon. (For that matter, Kolmogorov has strict priority on the optimal linear estimator for time-series. Uncle Norbert was unusually given to being a co-discoverer.) Given that a train averages so many spikes per second, it's a fairly straightforward exercise to compute the upper bound on how much information the train can carry, per second and per spike. These limits were first calculated by MacKay and McCulloch in 1954, and can't really be subverted --- the fewer constraints there are on possible patterns of spiking, the more distinguishable spike-trains, and so each train carries (on average) more information; by assuming only that the trains need to be consistent with the average rate, and so ignoring the refractory period and everything else, we're considering the least constrained ensemble of spike-trains, and so getting the physical upper limit on the rate at which the trains can convey information. The natural next question is to ask how close real neurons and their spikes get to this limit. Here there is a difficulty, because there is, surprisingly, no good way in general of measuring the rate of information transfer in most actual, physical channels. Our authors have, however, an ingenious way of underestimating it, using the measured statistics of the spikes and the first Wiener kernel. (Since this is an underestimate of the true rate of information transfer, and the MacKay-McCulloch limit is certainly an overestimate of what's physically possible, we know the true efficiency of the neural code is always greater than our estimate.) They then proceed to calculate the baud rate, bits per spike, and efficiency of various sensory and intermediate neurons; eye of newt is not exactly on the list, but retinal ganglion cells in the salamander (species unspecified) are. (A recent article adds two more types of neurons to the collection.) The baud rate varies from cell type to cell type from just under 50 bits/second to over 300 bits/second; the information per spike, however, is rather more constant, 2-3 bits/spike. The lowest estimated coding efficiency is 20% (for those salamander cells); most are closer to 50%. For the one case where naturalistic stimuli were provided, rather than just things which are easy to cobble up in the lab, the efficiency was about 90%. In other words: even when they're not doing the job they're designed for, many neurons come within a factor of two of the fundamental physical limits on their performance; they probably do much better on the stimuli for which they've adapted.

From here we got to some considerations on the reliability of neural computation. There's a classical theory, going back to von Neumann, of how to make reliable computers from unreliable parts, and it's been tacitly assumed, for the most part, that it applies to the nervous system. But those coding efficiencies should make us leery about thinking that neurons are especially flaky, and it seems that in some cases individual neurons are as reliable as the entire organism. The most impressive phenomena where this is not the case is hyper-acuity, where the organism is able to respond to differences which are smaller than what individual sensory cells should be able to detect, e.g., "echo-locating bats can apparent resolve jitter in the arrival time of their echoes with a precision of 10 nanoseconds" (p. 221, their italics). The trick here is indeed to use lots of neurons, but to discriminate between differences in patterns of activation from different stimuli, rather than averaging all of them. (As the authors point out, this is the same trick as the one that lets us see more than three colors!)

I'd have liked more at this point about how the neural code gets used in network computations, or, conversely, how to use the tricks we've learnt for breaking the code to get a handle on what the networks are computing and how, but this the authors largely reserve for future research (they have great hopes for the hippocampus and the motor cortex), along with a better understanding of the statistics of natural stimuli. They close with a "homage to the single spike," as a trustworthy and reliable carrier of a substantial amount of information, sometimes even responsible by itself for conscious sensations. "The individual spike, so often averaged in with its neighbors, deserves more respect."

This is obviously damn good science, and would make Spikes worth reading though its authors wrote like literary critics. Thankfully, they don't, and the book is a positive pleasure to read, clear, unpedantic, aware of the history on which it builds, and possessed of a restrained but noticeable and welcome sense of humor (e.g. citing George Carlin's "Seven Words You Can Never Say on Television"). Between the response functions, the fondness for working in the frequency domain rather than the time domain, and the references to Horowitz and Hill, physicists, electrical engineers, probabilists, etc. will feel quite at home; for the comfort of the less mathematically inclined among the biologists, there are ninety pages of "mathematical asides," together forming a decent course in information theory and time-series analysis accessible to anyone who remembers how to integrate, and there is not a single model. This is a really fun book which lays new paths across a very important field, and is accessible to almost anyone with an education in the natural sciences; in a word, a treasure.

xviii + 395 pp., diagrams, references, index
Neuroscience / Probability and Statistics
Currently in print as a hardback, ISBN 0-262-18174-6, US$95 [Buy from Powell's], and as a paperback, ISBN 0-262-68108-0, US$38 [Buy from Powell's]
11-12 May 1998 (minor revisions 13 Oct. 1999)