## Probability Theory

*18 Jul 2022 21:03*

One of my advisers in graduate school was a probability theorist, as was his adviser before him; I've not bothered to check, but I wouldn't be astonished if the chain went back to someone like Bernoulli. The fact that the chain could go back that far shows that mathematical probability is an old concept, almost as old as any other part of modern science; on the other hand, my adviser's adviser came just after the generation, between the wars, which made probability a respectable and rigorous branch of mathematics and removed countless obscurities from its applications. The first serious use of statistical methods in the sciences came only about a hundred years before that. Now of course error analysis is the first thing my students learn when they enter the lab. (Well, almost the first thing, after "if you don't write it down, it never happened" and "Cosma can be bribed with chocolate.") I am conditioned to attack every problem as some kind of stochastic process; but a few generations back nobody had any but the vaguest idea what a stochastic process was.

Pet peeves: Physicists who do not distinguish between a random variable ("X = the roll of a die") and the value it takes ("x=5"). People who report estimated numbers without error-bars or confidence-intervals. Bayesians.

Cf. math in general, stochastic processes, statistics, information theory, algorithmic information theory, statistical mechanics, ergodic theory, machine learning, statistical inference and induction, dynamics; large deviations; empirical process theory; concentration of measure; deviation inequalities; graph limits and exchangeable random graphs; Hilbert Space Methods for Statistics and Probability; projectivity in statistical models;

- Recommended, big-picture:
- Patrick Billingsley, Probability and Measure
- Harald Cramér, Mathematical Methods of Statistics [Review]
- Feller, An Introduction to Probability Theory and Its Applications, vol. I [I've not finished vol. II yet...]
- Bert Fristedt and Lawrence Gray, A Modern Approach to Probability Theory [Extremely thorough measure-theoretic text; nice treatment of stochastic processes]
- Geoffrey Grimmett and David Stirzaker, Probability and Random Processes [Maybe the best contemporary textbook for those who do not need measure-theoretic probability]
- Ian Hacking
- The Emergence of Probability [Where that strange two-faced notion came from, and why]
- The Taming of Chance [Putting chance to work in the 19th century]

- Mark Kac
- Engimas of Chance
- Probability and Related Topics in Physical Science
- Statistical Independence in Probability, Analysis and Number Theory

- Olav Kallenberg, Foundations of Modern Probability [My preferred textbook when teaching stochastic processes]
- Michel Loève, Probability Theory
- David Pollard, A User's Guide to Measure-Theoretic Probability
- R. F. Streater, "Classical and Quantum Probability," math-ph/0002049 ["There are few mathematical topics that are as badly taught to physicists as probability theory."]
- Aram Thomasian, The Structure of Probability Theory

- Recommended, close-ups (very miscellaneous):
- Philippe Barbe, "An Elementary Approach to Extreme Values Theory", arxiv:0811.0753
- Jochen Brocker, "A Lower Bound on Arbitrary
*f*-Divergences in Terms of the Total Variation" arxiv:0903.1765 - H. E. Daniels, "Mixtures of Geometric Distributions",
Journal of the Royal Statistical Society B
**23**(1961): 409--413 [JSTOR] - Clark Glymour, "Instrumental Probability", Monist
**84**(2001): 284--300 [PDF reprint] - Alexander E. Holroyd and Terry Soo, "A Non-Measurable Set from Coin-Flips", math.PR/0610705 [A cute construction to help students see the point of measure-theoretic probability]
- Mark Kac, Selected Papers
- Olav Kallenberg, Probabilitic Symmetries and Invariance
Principles [A tremendous book, but I must admit to a disappointment.
The three basic symmetries Kallenberg considers are symmetry under permutation (exchangeability), symmetry under rotation, and symmetry under "contraction" (i.e., integrating out variables). The obvious fourth is symmetry under
*translation*, or stationarity; this he frankly skips, on the grounds that so much has been written about it elsewhere. But I would very much like*his*take on it...] - Jill North, "Symmetry and Probability", phil-sci/2978
- Oystein Ore, Cardano, the Gambling Scholar [Includes a translation of Cardano's book on games of chance, which really does seem like the first appreciation of a quantitative sense of probability, a century before Pascal and Fermat (and apparently independent of them). Interestingly Cardano, at least in translation, speaks of "odds" or "chances" but not "probability".]
- Aris Spanos, "A frequentist interpretation of probability for
model-based inductive
inference", Synthese
**190**(2011) [With thanks to Prof. Spanos for letting me read a pre-publication draft] - Jakob Rosenthal, "The Natural-Range Conception of Probability", phil-sci/4978 [Defends the thesis that "the probability of an event is the proportion of initial states that lead to this event in the space of all possible initial states, provided that this proportion is approximately the same in any not too small interval of the initial state space.... [I]n the types of situations that give rise to probabilistic phenomena we may expect to find an initial state space such that any 'reasonable' density function over this space leads to the same probabilities for the possible outcomes."]

- To read, historical:
- William J. Adams, The Life and Times of the Central Limit Theorem
- Lorraine Daston, Classical Probability in the Enlightenment
- Hans Fischer, History of the Central Limit Theorem: From Laplace to Donsker
- Gerd Gigerenzer, Zeno Switjtink, Theodore Porter, Lorraine Daston, John Beatty and Lorenz Krüger, The Empire of Chance: How Probability Changed Science and Everyday Life
- Kendall and Plackett (eds.), Studies in the History of Statistics and Probability
- Andrei Kolmogorov, Foundations of Probability Theory
- Francesco Mainardi, Sergei Rogosin, "The origin of infinitely divisible distributions: from de Finetti's problem to Levy-Khintchine formula", arxiv:0801.1910
- Glenn Shafer and Vladimir Vovk, "The Sources of Kolmogorov's
Grundbegriffe", Statistical Science
**21**(2006): 70--98 = math.ST/0606533 - Reinhard Siegmund-Schultze
- "Probability in 1919/20: the von Mises-Pólya-Controversy", Archive for History of Exact Sciences
**60**(2006): 431--515 - "A Non-Conformist Longing for Unity in the Fractures of Modernity: Towards a Scientific Biography of Richard von Mises (1883--1953)",
Science in Context
**17**(2004): 333--370

- "Probability in 1919/20: the von Mises-Pólya-Controversy", Archive for History of Exact Sciences
- Jan von Plato, Creating Modern Probability

- To read, philosophical and foundational:
- Marshall Abrams, "Toward a Mechanistic Interpretation of Probability", phil-sci/4704
- Jordan Ellenberg and Elliott Sober, "Objective Probabilities in Number Theory" [PDF preprint]
- Eduardo M. R. A. Engel, A Road to Randomness in Physical Systems
- Alexander R. Pruss, "Probability, Regularity, and Cardinality",
Philosophy of Science
**80**(2013): 231--240 - John T. Roberts, "Laws About Frequencies", phil-sci/5058
- Michael Strevens, Bigger than Chaos: Understanding Complexity through Probability

- To read, pedagogical:
- Blom, Holst and Sandell, Problems and Snapshots from the World of Probability ["It is obvious that the authors have had fun in writing this book..."]
- F. M. Dekking, C. Kraaikamp, H. P. Lopuhaä and L. E. Meester, A Modern Introduction to Probability and Statistics: Understanding How and Why
- Feller, An Introduction to Probability Theory and Its Applications vol. II
- Allan Gut, Probability: A Graduate Course [From the back: "'I know it's trivial, but I have forgotten why'. This is a slightly exaggerated characterization of the unfortunate attitude of many mathematicians toward the surrounding world. The point of departure of this book is the opposite. This textbook on the theory of probability is aimed at graduate students, with the ideology that rather than being a purely mathematical discipline, probability theory is an intimate companion of statistics."]
- Svante Janson, "Probability asymptotics: notes on notation", arxiv:1108.3924 [Looks useful for the next time I teach stochastic processes]
- Emmanuel Lesigne, Heads or Tails: An Introduction to Limit Theorems in Probability
- Papoulis, Probability, Random Variables and Stochastic Processes
- Peter Olofsson, Probability, Statistics, and Stochastic Processes
- Sidney Resnick, A Probability Path
- A. Shiryaev, Probability Theory
- Stroock, Probability Theory: An Analytic View
- Paul Vitanyi, "Randomness," math.PR/0110086

- To read, technical:
- Sergio Albeverio and Song Liang, "Asymptotic expansions for the
Laplace approximations of sums of Banach space-valued random variables", Annals of
Probability
**33**(2005): 300--336 = math.PR/0503601 - David J. Aldous and Antar Bandyopadhyay, "A survey of max-type
recursive distributional equations", math.PR/0401388 = Annals of Applied
Probability
**15**(2005): 1047--1110 - David Balding, Pablo A. Ferrari, Ricardo Fraiman and Mariela Sued, "Limit theorems for sequences of random trees", math.PR/0406280 [Abstract: " We consider a random tree and introduce a metric in the space of trees to define the "mean tree" as the tree minimizing the average distance to the random tree. When the resulting metric space is compact we show laws of large numbers and central limit theorems for sequence of independent identically distributed random trees. As application we propose tests to check if two samples of random trees have the same law." I wonder if the same technique could be applied to other kinds of random graphs, e.g., random scale-free networks?]
- Patrick Billinglsey, Convergence of Probability Measures
- Salomon Bochner, Harmonic Analysis and the Theory of Probability
- Tapas Kumar Chandra, The Borel-Cantelli Lemma
- Louis H. Y. Chen, Larry Goldstein and Qi-Man Shao, Normal Approximation by Stein's Method
- I. Calvo, J. C. Cuchí, J. G. Esteve, F. Falceto, "Generalized Central Limit Theorem and Renormalization Group", arxiv:1009.2899
- Sourav Chatterjee, "A new method of normal approximation", arxiv:math/0611213
- Bernard Chazelle, The Discrepency Method: Randomness and Complexity
- Irene Crimaldi and Luca Pratelli, "Two inequalities for conditional
expectations and convergence results for filters", Statistics and
Probability Letters
**74**(2005): 151--162 - Victor De La Pena and Evarist Gine, Decoupling: From Dependence to Independence
- Victor H. de la Pena, Tze Leung Lai and Qi-Man Shan, Self-Normalized Processes: Limit Theory and Statistical Applications
- Janos Galambos and Italo Simonelli, Bonferroni-type Inequalities with Applications
- Stefano Galatolo, Mathieu Hoyrup, Cristobal Rojas, "A constructive Borel-Cantelli Lemma. Constructing orbits with required statistical properties", arxiv:0711.1478
- J. A. Gonzalez, L. I. Reyes, J. J. Suarez, L. E. Guerrero, and G. Gutierrez, "A mechanism for randomness," nlin.CD/0202022 [Color me skeptical, from the abstract]
- Martin Hairer, "A theory of regularity structures", arxiv:1303.5113
- Oliver Johnson and Andrew Barron, "Fisher Information inequalities and the Central Limit Theorem," math.PR/0111020
- Oliver Johnson and Richard Samworth, "Central Limit Theorem and convergence to stable laws in Mallows distance", math.PR/0406218
- Laurent Mazliak, "Poincarés Odds", arxiv:1211.5737
- Henry McKean, Probability: The Classical Limit Theorems
- National Research Council (USA), Probability and Algorithms [online]
- Peter Orbanz, "Projective limit random probabilities on Polish spaces", Electronic Journal of Statistics
**5**(2011): 1354--1373 - Giovanni Peccati and Murad S. Taqqu, "Moments, cumulants and diagram formulae for non-linear functionals of random measures", arxiv:0811/1726
- Iosif Pinelis, "Between Chebyshev and Cantelli", arxiv:1011.6065
- Chris Preston, "Some notes on standard Borel and related spaces", arxiv:0809.3066
- Revesz, The Laws of Large Numbers
- R. Schweizer and A. Sklar, Probabilistic Metric Spaces
- Glenn Shafer and Vladimir Vovk, Probability and Finance: It's Only a Game! [Yet Another Foundation of Probability, this time from game-theory.]
- Akimichi Takemura, Vladimir Vovk, Glenn Shafer, "The generality of the zero-one laws", arxiv:0803.3679
- Ramon van Handel, Probability in High Dimension [PDF lecture notes]
- Roman Vershynin, High-Dimensional Probability: An Introduction with Applications in Data Science