## Neural Coding

*02 Nov 2020 14:16*

And the statistics of neural spike trains more generally.

Since I've written about the neural coding problem, at great length, in my review of Spikes (see below), I won't repeat myself here.

*Things to try to understand:* Distributed and population codes.
How much can be understood about coding without also understanding computation?

*Things to do:* Causal-state reconstruction on real neural spike
data. (Done; see below.) Transducer state reconstruction; states of the
inferred transducer = classes of stimuli (+ internal histories) which make a
difference to the cell. The information coherence measure should indicate the
quantity of distributed information in spike-trains. Calculate for actual
neuronal circuits; does this interpretation make sense?

See also: Information Theory; Neural Modeling and Data Analysis; Stochastic Processes; Synchronization; Synchronization in Neural Systems; Time Series

- Recommended, big picture:
- Larry Abbott and Terry Sejnowski (eds.), Neural Codes and Distributed Representations
- Chris Eliasmith and Charles Anderson, Neural Engineering: Computation, Representation, and Dynamics in Neurobiological Systems
- Liam Paninski, Jonathan Pillow, and Jeremy Lewi, "Statistical models for neural encoding, decoding, and optimal stimulus design", to appear in P. Cisek, T. Drew and J. Kalaska (eds.), Computational Neuroscience: Progress in Brain Research [PDF preprint]
- Alexandre Pouget, Peter Dayan and Richard S. Zemel, "Inference and
Computation with Population Codes", Annual
Review of Neuroscience
**26**(2003): 381--401 - Fred Rieke, David Warland, Rob de Ruyter van Steveninck and William
Bialek, Spikes: Exploring the Neural Code
[Review: Cells That Go
*ping,*or, the Value of the Three-Bit Spike]

- Recommended, close-ups:
- Jose M. Amigo, Janusz Szczepanski, Elek Wajnryb and Maria
V. Sanchez-Vives, "Estimating the Entropy Rate of Spike Trains via Lempel-Ziv
Complexity",
Neural
Computation
**16**(2004): 717--736 [Normally, I have strong views on using Lempel-Ziv to measure entropy rates, but here they are using the 1976 Lempel-Ziv definitions, not the 1978 ones. The difference is subtle, but important; 1978 leads to gzip and practical compression algorithms, but very bad entropy estimates; 1976 leads, as they show numerically, to quite good entropy rate estimates, at least for some processes. Thanks to Dr. Szczepanski for correspondence about this paper.] - Riccardo Barbieri, Loren M. Frank, David P. Nguyen, Michael
C. Quirk, Victor Solo, Matthew A. Wilson and Emery N. Brown, "Dynamic Analyses
of Information Encoding in Neural
Ensembles", Neural
Computation
**16**(2004): 277--307 - M. J. Barber, J. W. Clark and C. H. Anderson, "Neural
Representation of Probabilistic Information," Neural Computation
**15**(2003): 1843--1864, cond-mat/0108425 - David Brillinger
- "Nerve Cell Spike Train Data Analysis: A Progression of
Technique," Journal of the American Statistical Association
**87**(1992): 260--270 - and Allessandro E. P. Villa, "Assessing Connections in Networks of Biological Neurons", pp. 77--92 in D. R. Brillinger, L. T. Fernholz and S. Morgenthaler (eds.), The Practice of Data Analysis: Essays in Honor of John W. Tukey [PS]

- "Nerve Cell Spike Train Data Analysis: A Progression of
Technique," Journal of the American Statistical Association
- A. E. Brockwell, A. L. Rojas and R. E. Kass, "Recursive
Bayesian Decoding of Motor Cortical Signals by Particle Filtering",
Journal of
Neurophysiology
**91**(2004): 1899--1907 [Very nice, especially since they've combining data from multiple experiments. It is a*little*disappointing that they set up a state-space model, but then only use the state to enforce a kind of weak continuity constraint on the decoding, rather than trying to capture the actual computations going on. But I should talk to them about that... Appendix A gives a very clear and compact explanation of particle filtering.] - Uri T. Eden, Loren M. Frank, Riccardo Barbieri, Victor Solo and
Emery N. Brown, "Dynamic Analysis of Neural Encoding by Point Process Adaptive
Filtering", Neural
Computation
**16**(2005): 971-988 - Nicol S. Harper and David McAlpine, "Optimal neural population
coding of an auditory spatial cue", Nature
**430**(2004): 682--686 - Jerome Y. Lettvin, H. R. Maturana, Warren S. McCulloch and
W. H. Pitts, "What the Frog's Eye Tells the Frog's Brain", Proceedings of
the IRE,
**47**(1959): 1940--1951 [Reprinted in McCulloch's Embodiments of Mind, among other places, and available online, e.g., here] - Thomas Naselaris, Kendrick N. Kay, Shinji Nishimoto and Jack L. Gallant, "Encoding and decoding in fMRI", Neuroimage
**56**(2011): 400--410, PMC3037423 - Jonathan W. Pillow, Yashar Ahmadian and Liam Paninski,
"Model-Based Decoding, Information Estimation, and Change-Point Detection Techniques for Multineuron Spike Trains", Neural Computation
**23**(2011): 1--45 - R. Quian Quiroga, L. Reddy, G. Kreiman, C. Koch and I. Fried,
"Invariant visual representation by single neurons in the human brain", Nature
**435**(2005): 1102--1107 - Tatyana Sharpee, Nicole C. Rust and William Bialek, "Analyzing
neural responses to natural signals: maximally informative dimensions", Neural Computation
**16**(2004): 223--250, physics/0212110 - S. P. Strong, Roland Koberle, Rob de Ruyter van Steveninck and
William Bialek, "Entropy and Information in Neural Spike Trains,"
Physical Review Letters
**80**(1998): 197--201 - Eric E. Thomson and William B. Kristan, "Quantifying Stimulus
Discriminability: A Comparison of Information Theory and Ideal Observer
Analysis",
Neural
Computation
**17**(2005): 741--778 [A useful warning against a too-common abuse of information theory. Thanks to Eric for providing me with a pre-print.] - Shreejoy J. Tripathy, Krishnan Padmanabhan, Richard C. Gerkin, and Nathaniel N. Urban, "Intermediate intrinsic diversity enhances neural population coding", Proceedings of the National Academy of Sciences (USA)
**110**(2013): 8248--8253 - Jonathan D. Victor and Keith P. Purpura, "Metric-Space Analysis of
Spike Trains: Theory, Algorithms and Application," Network: Computation
in Neural Systems
**8**(1997): 127--164 - Vincent Q. Vu, Bin Yu, Robert E. Kass, "Information In The Non-Stationary Case", arxiv:0806.3978 ### Modesty forbids me to recommend:
- Rob Haslinger, Kristina Klinkner and CRS, "The Computational Structure of Spike Trains", Neural Computation
**22**(2010): 121--157, arxiv:1001.0036 ### To read: - Craig A. Atencio, Tatyana O. Sharpee, and Christoph E. Schreiner, "Hierarchical computation in the canonical auditory cortical circuit", Proceedings of the National Academy of Sciences (USA)
**106**(2009): 21894--21899 - E. D. Adrian, Physical Background of Perception [Adrian was one of the first --- maybe the first? --- to record spike trains from neurons, and realize they were how neurons communicate]
- Blaise Aguera y Arcas and Adrienne Fairhall, "What causes a neuron to spike?" physics/0301014
- Blaise Aguera y Arcas, Adrienne L. Fairhall and William Bialek, "Computation in a single neuron: Hodgkin and Huxley revisited," physics/0212113
- Kazuyuki Aihara and Isao Tokuda, "Possible neural coding with
interevent intervals of synchronous firing," Physical Review
E
**66**(2002): 026212 - Vijay Balasubramanian and Michael J. Berry, "Metabolically Efficient Codes in The Retina," cond-mat/0105128
- Vijay Balasubramanian, Don Kimber and Michael J. Berry, "Metabolically Efficient Information Processing," cond-mat/0105127
- C. T. Bergstrom and M. Rosvall, "The transmission sense of information", arxiv:0810.4168
- John A. Berkowitz and Tatyana O. Sharpee, "Quantifying Information Conveyed by Large Neuronal Populations",
Neural Computation
**31**(2019): 1015--1047 - Michele Bezzi, Mathew E. Diamond and Alessandro Treves, "Redundency and synergy arising from correlations in large ensembles," cond-mat/0012119
- Michele Bezzi, Ines Samengo, Stefan Leutgeb and Sheri Mizumori, "Measuring information spatial densities," cond-mat/0111150
- William Bialek and Rob R. de Ruyter van Steveninck, "Features and dimensions: Motion estimation in fly vision", q-bio.NC/0505003
- Naama Brenner, Steven P. Strong, Roland Koberle, William Bialek and
Rob R. de Ruter van Steveninck, "Synergy in a Neural Code," Neural
Computation,
**12**(2000): 1531--1552 - Lars Buesing and Wolfgang Maass, "A Spiking Neuron as Information Bottleneck", Neural Computation
**22**(2010): 1961--1992 - Daniel A. Butts, Chong Weng, Jianzhong Jin, Chun-I Yeh, Nicholas A. Lesica, Jose-Manuel Alonso and Garrett B. Stanley, "Temporal precision in the neural code and the timescales of natural vision", Nature
**449**(2007): 92--95 - C. E. Carr and M. A. Friedman, "Evolution of Time Coding
Systems," Neural
Computation
**11**(1999): 1--20 - Guillermo A. Cecchi, Mariano Sigman, Josée-Manuel Alonso, Luis Martínez, Dante r. Chialvo and Marcelo O. Magnasco, "Noise in Neurons is Message-Dependent," cond-mat/0004492
- Mircea I. Chelaru and Valentin Dragoi, "Efficient coding in
heterogeneous neuronal
populations", Proceedings
of the National Academy of Sciences
**105**(2008): 16344--16349 - Yuzhi Chen, Wilson S. Geisler and Eyal Seidemann, "Optimal decoding of correlated neural population responses in the primate visual cortex",
Nature
Neuroscience
**9**(2006): 1412--1420 [This sounds cool, and of course I shouldn't comment before reading more than just the abstract, but of course I will anyway. "This optimal decoder consistently outperformed the monkey in the detection task, demonstrating the sensitivity of our techniques": yes, but doesn't that by the same token inidcate their irrelevance to understanding the*monkey's*neural code?] - Marshall Crumiller, Bruce Knight, Yunguo Yu and Ehud Kaplan, "Estimating the amount of information conveyed by a population of neurons" [PDF preprint via Dr. Kaplan]
- Isabel Dean, Nicol S Harper and David McAlpine, "Neural population
coding of sound level adapts to stimulus
statistics", Nature
Neuroscience
**8**(2005): 1684--1689 - Coralie de Hemptinne, Sylvie Nozaradan, Quentin Duvivier, Philippe
Lefevre, and Marcus Missal, "How Do Primates Anticipate Uncertain Future
Events?",
Journal of
Neuroscience
**27**(2007): 4334--4341 - Valeria Del Prete, "A replica free evaluation of the neuronal population information with mixed continuous and discrete stimuli: from the linear to the asymptotic regime," cond-mat/0301457
- David R. Euston and Bruce L. McNaughton, "Apparent Encoding of
Sequential Context in Rat Medial Prefrontal Cortex Is Accounted for by
Behavioral Variability", The Journal of
Neuroscience
**26**(2006): 13143--13155 - Hugo G. Eyherabide, Ariel Rokem, Andreas V. M. Herz, Ines Samengo, "Burst firing is a neural code in an insect auditory system", arxiv:0807.2550
- Adrienne L. Fairhall, Geofrrey D. Lewen, William Bialek and Robert
R. de Ruyter van Steveninck, "Efficiency and Ambiguity in an Adaptive Neural
Code," Nature
**412**(2001): 787--792 - Michael Famulare and Adrienne Fairhall, "Feature Selection in Simple Neurons: How Coding Depends on Spiking Dynamics", Neural Computation
**22**(2010): 581--598 - F. Gabbiani
- "Rate Coding and Signal Processing"
- and J. Midtgaard, "Neural Information Processing"

- Surya Ganguli, Dongsung Huh, and Haim Sompolinsky, "Memory traces
in dynamical systems", Proceedings of the National Academy of Sciences (USA)
**105**(2008): 18970--18975 - Yun Gao, Ioannis Kontoyiannis, Elie Bienenstock, "From the entropy to the statistical structure of spike trains", arxiv:0710.4117
- Ralf M. Haefner and Matthias Bethge, "Evaluating neuronal codes for inference using Fisher information", NIPS 23 (2010) [PDF]
- Yi Hao, Alon Orlitsky, "Data Amplification: Instance-Optimal Property Estimation", arxiv:1903.01432
- Kenneth D. Harris, "Neural Signatures of Cell Assembly
Organization", Nature Reviews
Neuroscience
**6**(2005): 399--407 - V. Hok, E. Save, P. P. Lenck-Santini and B. Poucet, "Coding for
spatial goals in the prelimbic/inframlimbic area of the rat frontal cortex", PNAS
**102**(2005): 4602--4607 - Toshihiko Hosoya, Stephen A. Baccus and Markus Meister, "Dynamic
predictive coding by the retina", Nature
**436**(2005): 71--77 - Wentao Huang and Kechen Zhang, "Information-Theoretic Bounds and Approximations in Neural Population Coding", Neural Computation
**30**(2018): 885--944 - Quentin J. M. Huys, Richard S. Zemel, Rama Natarajan, and Peter
Dayan , "Fast Population Coding", Neural
Computation
**19**(2007): 404--441 - Ole Jensen, "Information Transfer Between Rhythmically Coupled Networks: Reading the Hippocampal Phase Code," Neural Computation vol. 13 no. 12 (December 2001)
- Christof Koch, Biophysics of Computation: Information Processing in Single Neuron
- Philipp Knüsel, Reto Wyss, Peter König and Paul
F.M.J. Verschure, "Decoding a Temporal Population
Code", Neural
Computation
**16**(2004): 2079--2100 - R. Krahe, G. Kreiman, F. Gabbiani, C. Koch, W. Metzner, "Stimulus encoding and feature extraction by multiple sensory neurons" [Reprint]
- Nikolaus Kriegeskorte, Visual Population Codes: Towards a Common Multivariate Framework for Cell Recording and Functional Imaging
- Petr Lansky and Priscilla E. Greenwood, "Optimal Signal Estimation
in Neuronal
Models", Neural
Computation
**17**(2005): 2240--2257 - G. D. Lewen, W. Bialek and R. R. de Ruyter van Steveninck, "Neural coding of naturalistic motion stimuli," physics/0103088
- Longnian Lin, Remus Osan, Shy Shoham, Wenjun Jin, Wenqi Zuo, and
Joe Z. Tsien, "Identification of network-level coding units for real-time
representation of episodic experiences in the hippocampus", PNAS
**102**(2005): 6125--6130 - Christian K. Machens, "Adaptive sampling by information maximization," physics/0112070
- Gary Marsat and Gerald S. Pollack, "A Behavioral Role for Feature
Detection by Sensory
Bursts", The
Journal of Neuroscience
**26**(2006): 10542--10547 - Laura Martignon, Gustavo Deco, Kathryn Laskey, Mathew Diamond,
Winrich Freiwald and Eilon Vaadia, "Neural Coding: Higher-Order Temporal
Patterns in the Neurostatistics of Cell Assemblies," Neural
Computation
**12**(2000): 2621--2653 - Mark D. McDonnell, Nigel G. Stocks, Charles E. M. Pearce and Derek Abbott, "Point singularities and suprathreshold stochastic resonance in optimal coding", cond-mat/0409528
- Panzeri and Schultz, "A Unified Approach to the Study of Temporal,
Correlational, and Rate Coding," Neural
Computation
**13**(2001): 1311--1349 - Phillips and Singer, In Search of Common Foundations for Cortical Computation
- Jonathan W. Pillow, Liam Paninski, Valerie J. Uzzell, Eero
P. Simoncelli, and E. J. Chichilnisky, "Prediction and Decoding of Retinal
Ganglion Cell Responses with a Probabilistic Spiking
Model", The
Journal of Neuroscience
**25**(2005): 11003--11013 - Jonathan W. Pillow, Jonathon Shlens, Liam Paninski, Alexander Sher,
Alan M. Litke, E. J. Chichilnisky and Eero P. Simoncelli, "Spatio-temporal correlations and visual signalling in a complete neuronal population", Nature
**454**(2008): 995--999 - Alessio Plebe and Vivian M. De La Cruz, "Neural Representations Beyond ``Plus X''", Minds and Machines
**28**(2018): 93--117 - K. Prank, F. Gabbiani and G. Brabant, "Coding efficiency and information rates in transmembrane signaling" [Abstract]
- D. S. Reich, F. Mechler and J. D. Victor, "Independent and
Redundant Information in Nearby Cortical
Neurons", Science
**294**(2001): 2566--2568 - Hugh P. C. Robinson and Annette Harsch, "Stages of spike time
variability during neuronal responses to transient inputs," Physical
Review E
**66**(2002): 061902 - Enrico Rossoni and Jianfeng Feng, "Decoding spike train ensembles:
tracking a moving
stimulus", Biological
Cybernetics
**96**(2007): 99--112 [Improvements for some non-stationary situations through censored maximum likelihood estimation] - Rob de Ruyter van Steveninck and William Bialek, "Timing and Counting Precision in the Blowfly Visual System," physics/0202014
- Ines Samengo, "Information loss in an optimal maximum likelihood decoding," physics/0110074
- Elad Schneidman, William Bialek and Michael J. Berry, II, "An information theoretic approach to the functional classification of neurons," physics/0212114
- Maoz Shamir and Haim Sompolinsky, "Nonlinear Population Codes",
Neural
Computation
**16**(2004): 1105--1136 - Tatyana Sharpee and William Bialek, "Neural Decision Boundaries for Maximal Information Transmission", q-bio.NC/0703046
- Kyle H. Srivastava, Caroline M. Holmes, Michiel Vellema, Andrea R. Pack, Coen P. H. Elemans, Ilya Nemenman, and Samuel J. Sober, "Motor control by precisely timed spike patterns", Proceedings of the
National Academy of Sciences (USA)
**114**(2017): 1171--1176 - Richard B. Stein, E. Roderich Gossen and Kelvin E. Jones, "Neuronal
Variability: Noise or Part of the Signal?", Nature Reviews
Neuroscience
**6**(2005): 389--397 - Michael Stiber, "Spike timing precision and neural error correction: local behavior", q-bio.NC/0501021
- Giulio Tononi and Olaf Sporns, "Measuring information integration",
Biomedcentral
Neuroscience
**4**(2003): 31 [Really more neural information theory than neural*coding*as such] - Nicholas Watters and George N. Reeke, "Neuronal Spike Train Entropy Estimation by History Clustering", Neural Computation
**26**(2014): 1840--1872 - Brian D. Wright, Kamal Sen, William Bialek and Allison J. Doupe, "Spike timing and the coding of naturalistic sounds in a central auditory area of songbirds," physics/0201027
- Si Wu and Shun-ichi Amari, "Computing with Continuous Attractors: Stability and Online Aspects", Neural
Computation
**17**(2005); 2215--2239