Neural Coding

Last update: 09 Feb 2025 04:06
First version: 20 September 2001

And the statistics of neural spike trains more generally.

Since I've written about the neural coding problem, at great length, in my review of Spikes (see below), I won't repeat myself here.

Things to try to understand: Distributed and population codes. How much can be understood about coding without also understanding computation?

Things to do: Causal-state reconstruction on real neural spike data. (Done; see below.) Transducer state reconstruction; states of the inferred transducer = classes of stimuli (+ internal histories) which make a difference to the cell. The information coherence measure should indicate the quantity of distributed information in spike-trains. Calculate for actual neuronal circuits; does this interpretation make sense?

Larry Abbott and Terry Sejnowski (eds.), Neural Codes and Distributed Representations
Chris Eliasmith and Charles Anderson, Neural Engineering: Computation, Representation, and Dynamics in Neurobiological Systems
Liam Paninski, Jonathan Pillow, and Jeremy Lewi, "Statistical models for neural encoding, decoding, and optimal stimulus design", to appear in P. Cisek, T. Drew and J. Kalaska (eds.), Computational Neuroscience: Progress in Brain Research [PDF preprint]
Alexandre Pouget, Peter Dayan and Richard S. Zemel, "Inference and Computation with Population Codes", Annual Review of Neuroscience 26 (2003): 381--401
Fred Rieke, David Warland, Rob de Ruyter van Steveninck and William Bialek, Spikes: Exploring the Neural Code [Review: Cells That Go ping, or, the Value of the Three-Bit Spike]

Jose M. Amigo, Janusz Szczepanski, Elek Wajnryb and Maria V. Sanchez-Vives, "Estimating the Entropy Rate of Spike Trains via Lempel-Ziv Complexity", Neural Computation 16 (2004): 717--736 [Normally, I have strong views on using Lempel-Ziv to measure entropy rates, but here they are using the 1976 Lempel-Ziv definitions, not the 1978 ones. The difference is subtle, but important; 1978 leads to gzip and practical compression algorithms, but very bad entropy estimates; 1976 leads, as they show numerically, to quite good entropy rate estimates, at least for some processes. Thanks to Dr. Szczepanski for correspondence about this paper.]
Riccardo Barbieri, Loren M. Frank, David P. Nguyen, Michael C. Quirk, Victor Solo, Matthew A. Wilson and Emery N. Brown, "Dynamic Analyses of Information Encoding in Neural Ensembles", Neural Computation 16 (2004): 277--307
M. J. Barber, J. W. Clark and C. H. Anderson, "Neural Representation of Probabilistic Information," Neural Computation 15 (2003): 1843--1864, cond-mat/0108425
David Brillinger
- "Nerve Cell Spike Train Data Analysis: A Progression of Technique," Journal of the American Statistical Association 87 (1992): 260--270
- and Allessandro E. P. Villa, "Assessing Connections in Networks of Biological Neurons", pp. 77--92 in D. R. Brillinger, L. T. Fernholz and S. Morgenthaler (eds.), The Practice of Data Analysis: Essays in Honor of John W. Tukey [PS]
A. E. Brockwell, A. L. Rojas and R. E. Kass, "Recursive Bayesian Decoding of Motor Cortical Signals by Particle Filtering", Journal of Neurophysiology 91 (2004): 1899--1907 [Very nice, especially since they've combining data from multiple experiments. It is a little disappointing that they set up a state-space model, but then only use the state to enforce a kind of weak continuity constraint on the decoding, rather than trying to capture the actual computations going on. But I should talk to them about that... Appendix A gives a very clear and compact explanation of particle filtering.]
Uri T. Eden, Loren M. Frank, Riccardo Barbieri, Victor Solo and Emery N. Brown, "Dynamic Analysis of Neural Encoding by Point Process Adaptive Filtering", Neural Computation 16 (2005): 971-988
Nicol S. Harper and David McAlpine, "Optimal neural population coding of an auditory spatial cue", Nature 430 (2004): 682--686
Jerome Y. Lettvin, H. R. Maturana, Warren S. McCulloch and W. H. Pitts, "What the Frog's Eye Tells the Frog's Brain", Proceedings of the IRE, 47 (1959): 1940--1951 [Reprinted in McCulloch's Embodiments of Mind, among other places, and available online, e.g., here]
Thomas Naselaris, Kendrick N. Kay, Shinji Nishimoto and Jack L. Gallant, "Encoding and decoding in fMRI", Neuroimage 56 (2011): 400--410, PMC3037423
Jonathan W. Pillow, Yashar Ahmadian and Liam Paninski, "Model-Based Decoding, Information Estimation, and Change-Point Detection Techniques for Multineuron Spike Trains", Neural Computation 23 (2011): 1--45
R. Quian Quiroga, L. Reddy, G. Kreiman, C. Koch and I. Fried, "Invariant visual representation by single neurons in the human brain", Nature 435 (2005): 1102--1107
Tatyana Sharpee, Nicole C. Rust and William Bialek, "Analyzing neural responses to natural signals: maximally informative dimensions", Neural Computation 16 (2004): 223--250, physics/0212110
S. P. Strong, Roland Koberle, Rob de Ruyter van Steveninck and William Bialek, "Entropy and Information in Neural Spike Trains," Physical Review Letters 80 (1998): 197--201
Eric E. Thomson and William B. Kristan, "Quantifying Stimulus Discriminability: A Comparison of Information Theory and Ideal Observer Analysis", Neural Computation 17 (2005): 741--778 [A useful warning against a too-common abuse of information theory. Thanks to Eric for providing me with a pre-print.]
Shreejoy J. Tripathy, Krishnan Padmanabhan, Richard C. Gerkin, and Nathaniel N. Urban, "Intermediate intrinsic diversity enhances neural population coding", Proceedings of the National Academy of Sciences (USA) 110 (2013): 8248--8253
Jonathan D. Victor and Keith P. Purpura, "Metric-Space Analysis of Spike Trains: Theory, Algorithms and Application," Network: Computation in Neural Systems 8 (1997): 127--164
Vincent Q. Vu, Bin Yu, Robert E. Kass, "Information In The Non-Stationary Case", arxiv:0806.3978 ### Modesty forbids me to recommend:
Rob Haslinger, Kristina Klinkner and CRS, "The Computational Structure of Spike Trains", Neural Computation 22 (2010): 121--157, arxiv:1001.0036 ### To read:
Craig A. Atencio, Tatyana O. Sharpee, and Christoph E. Schreiner, "Hierarchical computation in the canonical auditory cortical circuit", Proceedings of the National Academy of Sciences (USA) 106 (2009): 21894--21899
E. D. Adrian, Physical Background of Perception [Adrian was one of the first --- maybe the first? --- to record spike trains from neurons, and realize they were how neurons communicate]
Blaise Aguera y Arcas and Adrienne Fairhall, "What causes a neuron to spike?" physics/0301014
Blaise Aguera y Arcas, Adrienne L. Fairhall and William Bialek, "Computation in a single neuron: Hodgkin and Huxley revisited," physics/0212113
Kazuyuki Aihara and Isao Tokuda, "Possible neural coding with interevent intervals of synchronous firing," Physical Review E 66 (2002): 026212
Vijay Balasubramanian and Michael J. Berry, "Metabolically Efficient Codes in The Retina," cond-mat/0105128
Vijay Balasubramanian, Don Kimber and Michael J. Berry, "Metabolically Efficient Information Processing," cond-mat/0105127
C. T. Bergstrom and M. Rosvall, "The transmission sense of information", arxiv:0810.4168
John A. Berkowitz and Tatyana O. Sharpee, "Quantifying Information Conveyed by Large Neuronal Populations", Neural Computation 31 (2019): 1015--1047
Michele Bezzi, Mathew E. Diamond and Alessandro Treves, "Redundency and synergy arising from correlations in large ensembles," cond-mat/0012119
Michele Bezzi, Ines Samengo, Stefan Leutgeb and Sheri Mizumori, "Measuring information spatial densities," cond-mat/0111150
William Bialek and Rob R. de Ruyter van Steveninck, "Features and dimensions: Motion estimation in fly vision", q-bio.NC/0505003
Naama Brenner, Steven P. Strong, Roland Koberle, William Bialek and Rob R. de Ruter van Steveninck, "Synergy in a Neural Code," Neural Computation, 12 (2000): 1531--1552
Lars Buesing and Wolfgang Maass, "A Spiking Neuron as Information Bottleneck", Neural Computation 22 (2010): 1961--1992
Daniel A. Butts, Chong Weng, Jianzhong Jin, Chun-I Yeh, Nicholas A. Lesica, Jose-Manuel Alonso and Garrett B. Stanley, "Temporal precision in the neural code and the timescales of natural vision", Nature 449 (2007): 92--95
C. E. Carr and M. A. Friedman, "Evolution of Time Coding Systems," Neural Computation 11 (1999): 1--20
Guillermo A. Cecchi, Mariano Sigman, Josée-Manuel Alonso, Luis Martínez, Dante r. Chialvo and Marcelo O. Magnasco, "Noise in Neurons is Message-Dependent," cond-mat/0004492
Mircea I. Chelaru and Valentin Dragoi, "Efficient coding in heterogeneous neuronal populations", Proceedings of the National Academy of Sciences 105 (2008): 16344--16349
Yuzhi Chen, Wilson S. Geisler and Eyal Seidemann, "Optimal decoding of correlated neural population responses in the primate visual cortex", Nature Neuroscience 9 (2006): 1412--1420 [This sounds cool, and of course I shouldn't comment before reading more than just the abstract, but of course I will anyway. "This optimal decoder consistently outperformed the monkey in the detection task, demonstrating the sensitivity of our techniques": yes, but doesn't that by the same token inidcate their irrelevance to understanding the monkey's neural code?]
Marshall Crumiller, Bruce Knight, Yunguo Yu and Ehud Kaplan, "Estimating the amount of information conveyed by a population of neurons" [PDF preprint via Dr. Kaplan]
Isabel Dean, Nicol S Harper and David McAlpine, "Neural population coding of sound level adapts to stimulus statistics", Nature Neuroscience 8 (2005): 1684--1689
Coralie de Hemptinne, Sylvie Nozaradan, Quentin Duvivier, Philippe Lefevre, and Marcus Missal, "How Do Primates Anticipate Uncertain Future Events?", Journal of Neuroscience 27 (2007): 4334--4341
Valeria Del Prete, "A replica free evaluation of the neuronal population information with mixed continuous and discrete stimuli: from the linear to the asymptotic regime," cond-mat/0301457
David R. Euston and Bruce L. McNaughton, "Apparent Encoding of Sequential Context in Rat Medial Prefrontal Cortex Is Accounted for by Behavioral Variability", The Journal of Neuroscience 26 (2006): 13143--13155
Hugo G. Eyherabide, Ariel Rokem, Andreas V. M. Herz, Ines Samengo, "Burst firing is a neural code in an insect auditory system", arxiv:0807.2550
Adrienne L. Fairhall, Geofrrey D. Lewen, William Bialek and Robert R. de Ruyter van Steveninck, "Efficiency and Ambiguity in an Adaptive Neural Code," Nature 412 (2001): 787--792
Michael Famulare and Adrienne Fairhall, "Feature Selection in Simple Neurons: How Coding Depends on Spiking Dynamics", Neural Computation 22 (2010): 581--598
F. Gabbiani
- "Rate Coding and Signal Processing"
- and J. Midtgaard, "Neural Information Processing"
Surya Ganguli, Dongsung Huh, and Haim Sompolinsky, "Memory traces in dynamical systems", Proceedings of the National Academy of Sciences (USA) 105 (2008): 18970--18975
Yun Gao, Ioannis Kontoyiannis, Elie Bienenstock, "From the entropy to the statistical structure of spike trains", arxiv:0710.4117
Ralf M. Haefner and Matthias Bethge, "Evaluating neuronal codes for inference using Fisher information", NIPS 23 (2010) [PDF]
Yi Hao, Alon Orlitsky, "Data Amplification: Instance-Optimal Property Estimation", arxiv:1903.01432
Kenneth D. Harris, "Neural Signatures of Cell Assembly Organization", Nature Reviews Neuroscience 6 (2005): 399--407
V. Hok, E. Save, P. P. Lenck-Santini and B. Poucet, "Coding for spatial goals in the prelimbic/inframlimbic area of the rat frontal cortex", PNAS 102 (2005): 4602--4607
Toshihiko Hosoya, Stephen A. Baccus and Markus Meister, "Dynamic predictive coding by the retina", Nature 436 (2005): 71--77
Wentao Huang and Kechen Zhang, "Information-Theoretic Bounds and Approximations in Neural Population Coding", Neural Computation 30 (2018): 885--944
Quentin J. M. Huys, Richard S. Zemel, Rama Natarajan, and Peter Dayan , "Fast Population Coding", Neural Computation 19 (2007): 404--441
Ole Jensen, "Information Transfer Between Rhythmically Coupled Networks: Reading the Hippocampal Phase Code," Neural Computation vol. 13 no. 12 (December 2001)
Christof Koch, Biophysics of Computation: Information Processing in Single Neuron
Philipp Knüsel, Reto Wyss, Peter König and Paul F.M.J. Verschure, "Decoding a Temporal Population Code", Neural Computation 16 (2004): 2079--2100
R. Krahe, G. Kreiman, F. Gabbiani, C. Koch, W. Metzner, "Stimulus encoding and feature extraction by multiple sensory neurons" [Reprint]
Nikolaus Kriegeskorte, Visual Population Codes: Towards a Common Multivariate Framework for Cell Recording and Functional Imaging
Petr Lansky and Priscilla E. Greenwood, "Optimal Signal Estimation in Neuronal Models", Neural Computation 17 (2005): 2240--2257
G. D. Lewen, W. Bialek and R. R. de Ruyter van Steveninck, "Neural coding of naturalistic motion stimuli," physics/0103088
Longnian Lin, Remus Osan, Shy Shoham, Wenjun Jin, Wenqi Zuo, and Joe Z. Tsien, "Identification of network-level coding units for real-time representation of episodic experiences in the hippocampus", PNAS 102 (2005): 6125--6130
Christian K. Machens, "Adaptive sampling by information maximization," physics/0112070
Gary Marsat and Gerald S. Pollack, "A Behavioral Role for Feature Detection by Sensory Bursts", The Journal of Neuroscience 26 (2006): 10542--10547
Laura Martignon, Gustavo Deco, Kathryn Laskey, Mathew Diamond, Winrich Freiwald and Eilon Vaadia, "Neural Coding: Higher-Order Temporal Patterns in the Neurostatistics of Cell Assemblies," Neural Computation 12 (2000): 2621--2653
Mark D. McDonnell, Nigel G. Stocks, Charles E. M. Pearce and Derek Abbott, "Point singularities and suprathreshold stochastic resonance in optimal coding", cond-mat/0409528
Panzeri and Schultz, "A Unified Approach to the Study of Temporal, Correlational, and Rate Coding," Neural Computation 13 (2001): 1311--1349
Phillips and Singer, In Search of Common Foundations for Cortical Computation
Jonathan W. Pillow, Liam Paninski, Valerie J. Uzzell, Eero P. Simoncelli, and E. J. Chichilnisky, "Prediction and Decoding of Retinal Ganglion Cell Responses with a Probabilistic Spiking Model", The Journal of Neuroscience 25 (2005): 11003--11013
Jonathan W. Pillow, Jonathon Shlens, Liam Paninski, Alexander Sher, Alan M. Litke, E. J. Chichilnisky and Eero P. Simoncelli, "Spatio-temporal correlations and visual signalling in a complete neuronal population", Nature 454 (2008): 995--999
Alessio Plebe and Vivian M. De La Cruz, "Neural Representations Beyond ``Plus X''", Minds and Machines 28 (2018): 93--117
K. Prank, F. Gabbiani and G. Brabant, "Coding efficiency and information rates in transmembrane signaling" [Abstract]
D. S. Reich, F. Mechler and J. D. Victor, "Independent and Redundant Information in Nearby Cortical Neurons", Science 294 (2001): 2566--2568
Hugh P. C. Robinson and Annette Harsch, "Stages of spike time variability during neuronal responses to transient inputs," Physical Review E 66 (2002): 061902
Enrico Rossoni and Jianfeng Feng, "Decoding spike train ensembles: tracking a moving stimulus", Biological Cybernetics 96 (2007): 99--112 [Improvements for some non-stationary situations through censored maximum likelihood estimation]
Rob de Ruyter van Steveninck and William Bialek, "Timing and Counting Precision in the Blowfly Visual System," physics/0202014
Ines Samengo, "Information loss in an optimal maximum likelihood decoding," physics/0110074
Elad Schneidman, William Bialek and Michael J. Berry, II, "An information theoretic approach to the functional classification of neurons," physics/0212114
Maoz Shamir and Haim Sompolinsky, "Nonlinear Population Codes", Neural Computation 16 (2004): 1105--1136
Tatyana Sharpee and William Bialek, "Neural Decision Boundaries for Maximal Information Transmission", q-bio.NC/0703046
Kyle H. Srivastava, Caroline M. Holmes, Michiel Vellema, Andrea R. Pack, Coen P. H. Elemans, Ilya Nemenman, and Samuel J. Sober, "Motor control by precisely timed spike patterns", Proceedings of the National Academy of Sciences (USA) 114 (2017): 1171--1176
Richard B. Stein, E. Roderich Gossen and Kelvin E. Jones, "Neuronal Variability: Noise or Part of the Signal?", Nature Reviews Neuroscience 6 (2005): 389--397
Michael Stiber, "Spike timing precision and neural error correction: local behavior", q-bio.NC/0501021
Giulio Tononi and Olaf Sporns, "Measuring information integration", Biomedcentral Neuroscience 4 (2003): 31 [Really more neural information theory than neural coding as such]
Nicholas Watters and George N. Reeke, "Neuronal Spike Train Entropy Estimation by History Clustering", Neural Computation 26 (2014): 1840--1872
Brian D. Wright, Kamal Sen, William Bialek and Allison J. Doupe, "Spike timing and the coding of naturalistic sounds in a central auditory area of songbirds," physics/0201027
Si Wu and Shun-ichi Amari, "Computing with Continuous Attractors: Stability and Online Aspects", Neural Computation 17 (2005); 2215--2239