Notebooks
http://bactra.org/notebooks
Cosma's NotebooksenProbability Theory
http://bactra.org/notebooks/2022/07/18#probability
<P>One of my advisers in graduate school was a probability theorist, as was his
adviser before him; I've not bothered to check, but I wouldn't be astonished if
the chain went back to someone like Bernoulli. The fact that the chain could
go back that far shows that mathematical probability is an old concept, almost
as old as any other part of modern science; on the other hand, my adviser's
adviser came just after the generation, between the wars, which made
probability a respectable and rigorous branch of mathematics and removed
countless obscurities from its applications. The first serious use of
statistical methods in the sciences came only about a hundred years before
that. Now of course error analysis is the first thing my students learn when
they enter the lab. (Well, almost the first thing, after "if you don't write
it down, it never happened" and "Cosma can be bribed with chocolate.") I am
conditioned to attack every problem as some kind of stochastic process; but a
few generations back nobody had any but the vaguest idea what a stochastic
process was.
<P>Pet peeves: Physicists who do not distinguish between a random variable ("X
= the roll of a die") and the value it takes ("x=5"). People who report
estimated numbers without error-bars or confidence-intervals. Bayesians.
<P>Cf. <a href="math.html">math in general</a>,
<a href="stochastic-processes.html">stochastic processes</a>,
<a href="statistics.html">statistics</a>,
<a href="information-theory.html">information theory</a>,
<a href="algorithmic-information-theory.html">algorithmic information theory</a>,
<a href="stat-mech.html">statistical mechanics</a>,
<a href="ergodic-theory.html">ergodic theory</a>,
<a href="learning-inference-induction.html">machine learning,
statistical inference and induction</a>,
<a href="chaos.html">dynamics</a>;
<a href="large-deviations.html">large deviations</a>;
<a href="empirical-process-theory.html">empirical process theory</a>;
<a href="concentration-of-measure.html">concentration of measure</a>;
<a href="deviation-inequalities.html">deviation inequalities</a>;
<a href="graph-limits.html">graph limits and exchangeable random graphs</a>;
<a href="hilbert-space-for-stats.html">Hilbert Space Methods for Statistics and Probability</a>;
<a href="projectivity.html">projectivity in statistical models</a>;
<ul>Recommended, big-picture:
<li>Patrick Billingsley, <cite>Probability and Measure</cite>
<li>Harald Cramér, <cite>Mathematical Methods of
Statistics</cite> [<a href="../reviews/cramer-on-math-stat/">Review</a>]
<li>Feller, <cite>An Introduction to Probability Theory and Its
Applications</cite>, vol. I [I've not finished vol. II yet...]
<li>Bert Fristedt and Lawrence Gray, <cite>A Modern Approach to
Probability Theory</cite> [Extremely thorough measure-theoretic text; nice
treatment of stochastic processes]
<li>Geoffrey Grimmett and David Stirzaker, <cite>Probability and Random
Processes</cite> [Maybe the best contemporary textbook for those who do not
need measure-theoretic probability]
<li>Ian Hacking
<ul>
<li><cite>The Emergence of Probability</cite> [Where that
strange two-faced notion came from, and why]
<li><cite>The Taming of Chance</cite> [Putting chance to work
in the 19th century]
</ul>
<li>Mark Kac
<ul>
<li><cite>Engimas of Chance</cite>
<li><cite>Probability and Related Topics in Physical
Science</cite>
<li><cite>Statistical Independence in Probability, Analysis and
Number Theory</cite>
</ul>
<li>Olav Kallenberg, <cite>Foundations of Modern Probability</cite>
[My preferred textbook when teaching stochastic processes]
<li>Michel Loève, <cite>Probability Theory</cite>
<li>David Pollard, <cite>A User's Guide to Measure-Theoretic
Probability</cite>
<li><a href="http://www.mth.kcl.ac.uk/~streater/">R. F. Streater</a>,
"Classical and Quantum Probability,"
<a href="http://arxiv.org/abs/math-ph/0002049">math-ph/0002049</a> ["There are
few mathematical topics that are as badly taught to physicists as probability
theory."]
<li>Aram Thomasian, <cite>The Structure of Probability Theory</cite>
</ul>
<ul>Recommended, close-ups (very miscellaneous):
<li>Philippe Barbe, "An Elementary Approach to Extreme
Values Theory", <a href="http://arxiv.org/abs/0811.0753">arxiv:0811.0753</a>
<li>Jochen Brocker, "A Lower Bound on Arbitrary <em>f</em>-Divergences in
Terms of the Total Variation" <a href="http://arxiv.org/abs/0903.1765">arxiv:0903.1765</a>
<li>H. E. Daniels, "Mixtures of Geometric Distributions",
<cite>Journal of the Royal Statistical Society</cite> B <strong>23</strong> (1961): 409--413 [<a href="http://www.jstor.org/stable/2984030">JSTOR</a>]
<li>Clark Glymour, "Instrumental Probability", <cite>Monist</cite>
<strong>84</strong> (2001): 284--300 [<a href="http://www.hss.cmu.edu/philosophy/glymour/glymour2001.pdf">PDF reprint</a>]
<li>Alexander E. Holroyd and Terry Soo, "A Non-Measurable Set from
Coin-Flips", <a href="http://arxiv.org/abs/math.PR/0610705">math.PR/0610705</a>
[A cute construction to help students see the point of measure-theoretic
probability]
<li>Mark Kac, <cite>Selected Papers</cite>
<li>Olav Kallenberg, <cite>Probabilitic Symmetries and Invariance
Principles</cite> [A tremendous book, but I must admit to a disappointment.
The three basic symmetries Kallenberg considers are symmetry under permutation (exchangeability), symmetry under rotation, and symmetry under "contraction" (i.e., integrating out variables). The obvious fourth is symmetry under
<em>translation</em>, or stationarity; this he frankly skips, on the grounds
that so much has been written about it elsewhere. But I would very much like
<em>his</em> take on it...]
<li>Jill North, "Symmetry and Probability", <a
href="http://philsci-archive.pitt.edu/archive/00002978/">phil-sci/2978</a>
<li>Oystein Ore, <cite>Cardano, the Gambling Scholar</cite> [Includes a translation of Cardano's book on games of chance, which really does seem like
the first appreciation of a quantitative sense of probability, a century before Pascal and Fermat (and apparently independent of them). Interestingly Cardano, at least in translation, speaks of "odds" or "chances" but not "probability".]
<li>Aris Spanos, "A frequentist interpretation of probability for
model-based inductive
inference", <a href="http://dx.doi.org/10.1007/s11229-011-9892-x"><cite>Synthese</cite> <strong>190</strong>
(2011)</a> [With thanks to Prof. Spanos for letting me read a pre-publication draft]
<li>Jakob Rosenthal, "The Natural-Range Conception of Probability",
<a href="http://philsci-archive.pitt.edu/archive/00004978/">phil-sci/4978</a>
[Defends the thesis that "the probability of an event is the proportion of
initial states that lead to this event in the space of all possible initial
states, provided that this proportion is approximately the same in any not too
small interval of the initial state space.... [I]n the types of situations that
give rise to probabilistic phenomena we may expect to find an initial state
space such that any 'reasonable' density function over this space leads to the
same probabilities for the possible outcomes."]
</ul>
<ul>To read, historical:
<li>William J. Adams, <cite>The Life and Times of the Central
Limit Theorem</cite>
<li>Lorraine Daston, <cite><a
href="http://pup.princeton.edu/titles/4295.html">Classical Probability in the
Enlightenment</a></cite>
<li>Hans Fischer, <cite><a href="http://www.springer.com/mathematics/history+of+mathematics/book/978-0-387-87856-0">History of the Central Limit Theorem: From Laplace to Donsker</a></cite>
<li>Gerd Gigerenzer, Zeno Switjtink, Theodore Porter, Lorraine Daston,
John Beatty and Lorenz Krüger, <cite>The Empire of Chance: How Probability
Changed Science and Everyday Life</cite>
<li>Kendall and Plackett (eds.), <cite>Studies in the History of
Statistics and Probability</cite>
<li>Andrei Kolmogorov, <cite>Foundations of Probability Theory</cite>
<li>Francesco Mainardi, Sergei Rogosin, "The origin of infinitely divisible distributions: from de Finetti's problem to Levy-Khintchine formula", <a href="http://arxiv.org/abs/0801.1910">arxiv:0801.1910</a>
<li>Glenn Shafer and Vladimir Vovk, "The Sources of Kolmogorov's
Grundbegriffe", <cite>Statistical Science</cite> <strong>21</strong> (2006):
70--98 = <a href="http://arxiv.org/abs/math.ST/0606533">math.ST/0606533</a>
<li>Reinhard Siegmund-Schultze
<ul>
<li>"Probability in 1919/20: the von Mises-Pólya-Controversy", <a href="http://dx.doi.org/10.1007/s00407-006-0112-x"><cite>Archive for History of Exact Sciences</cite> <strong>60</strong> (2006): 431--515</a>
<li>"A Non-Conformist Longing for Unity in the Fractures of Modernity: Towards a Scientific Biography of Richard von Mises (1883--1953)",
<a href="http://dx.doi.org/10.1017/S026988970400016X"><cite>Science in Context</cite> <strong>17</strong> (2004): 333--370</a>
</ul>
<li>Jan von Plato, <cite>Creating Modern Probability</cite>
</ul>
<ul>To read, philosophical and foundational:
<li>Marshall Abrams, "Toward a Mechanistic Interpretation of Probability", <a href="http://philsci-archive.pitt.edu/4704">phil-sci/4704</a>
<li><a href="http://quomodocumque.wordpress.com/">Jordan Ellenberg</a> and <a href="http://philosophy.wisc.edu/sober/">Elliott Sober</a>,
"Objective Probabilities in Number Theory" [<a href="http://philosophy.wisc.edu/sober/Ellenberg%20and%20Sober%20Probability%20statements%20in%20number%20theory%20may%2031%202011.pdf">PDF preprint</a>]
<li>Eduardo M. R. A. Engel, <cite>A Road to Randomness in Physical
Systems</cite>
<li>Alexander R. Pruss, "Probability, Regularity, and Cardinality",
<a href="http://dx.doi.org/10.1086/670299"><cite>Philosophy of Science</cite> <strong>80</strong> (2013): 231--240</a>
<li>John T. Roberts, "Laws About Frequencies", <a href="http://philsci-archive.pitt.edu/5058/">phil-sci/5058</a>
<li>Michael Strevens, <cite>Bigger than Chaos: Understanding Complexity
through Probability</cite>
</ul>
<ul>To read, pedagogical:
<li>Blom, Holst and Sandell, <cite>Problems and Snapshots from the
World of Probability</cite> ["It is obvious that the authors have had fun in
writing this book..."]
<li>F. M. Dekking, C. Kraaikamp, H. P. Lopuhaä and L. E. Meester,
<cite><a
href="http://www.springeronline.com/sgw/cda/frontpage/0,11855,5-0-22-34951942-0,00.html">A Modern Introduction to Probability and Statistics: Understanding How
and Why</a></cite>
<li>Feller, <cite>An Introduction to Probability Theory and Its
Applications</cite> vol. II
<li>Allan Gut, <citE><a
href="http://www.springeronline.com/sgw/cda/frontpage/0,11855,5-0-22-34953310-0,00.html">Probability: A Graduate Course</a></cite> [From the
back: "'I know it's trivial, but I have forgotten why'. This is a slightly
exaggerated characterization of the unfortunate attitude of many mathematicians
toward the surrounding world. The point of departure of this book is the
opposite. This textbook on the theory of probability is aimed at graduate
students, with the ideology that rather than being a purely mathematical
discipline, probability theory is an intimate companion of statistics."]
<li>Svante Janson, "Probability asymptotics: notes on notation", <a href="http://arxiv.org/abs/1108.3924">arxiv:1108.3924</a> [Looks useful for the next time I teach stochastic processes]
<li>Emmanuel Lesigne, <cite><a
href="http://www.ams.org/bookstore?fn=20&arg1=probability&item=STML-28">Heads or Tails: An Introduction to Limit
Theorems in Probability</a></cite>
<li>Papoulis, <cite>Probability, Random Variables and Stochastic
Processes</cite>
<li>Peter Olofsson, <cite>Probability, Statistics, and Stochastic
Processes</cite>
<li>Sidney Resnick, <cite>A Probability Path</cite>
<li>A. Shiryaev, <cite>Probability Theory</cite>
<li>Stroock, <cite>Probability Theory: An Analytic View</cite>
<li>Paul Vitanyi, "Randomness," <a
href="http://arxiv.org/abs/math.PR/0110086">math.PR/0110086</a>
</ul>
<ul>To read, technical:
<li>Sergio Albeverio and Song Liang, "Asymptotic expansions for the
Laplace approximations of sums of Banach space-valued random variables", <a
href="http://dx.doi.org/10%2E1214/009117904000001017"><cite>Annals of
Probability</cite> <strong>33</strong> (2005): 300--336</a> = <a
href="http://arxiv.org/abs/math.PR/0503601">math.PR/0503601</a>
<li>David J. Aldous and Antar Bandyopadhyay, "A survey of max-type
recursive distributional equations", <a
href="http://arxiv.org/abs/math.PR/0401388">math.PR/0401388</a> = <a
href="http://dx.doi.org/10%2E1214/105051605000000142"><cite>Annals of Applied
Probability</cite> <strong>15</strong> (2005): 1047--1110</a>
<li>David Balding, Pablo A. Ferrari, Ricardo Fraiman and Mariela Sued,
"Limit theorems for sequences of random trees", <a
href="http://arxiv.org/abs/math.PR/0406280">math.PR/0406280</a> [Abstract: " We
consider a random tree and introduce a metric in the space of trees to define
the "mean tree" as the tree minimizing the average distance to the random
tree. When the resulting metric space is compact we show laws of large numbers
and central limit theorems for sequence of independent identically distributed
random trees. As application we propose tests to check if two samples of random
trees have the same law." I wonder if the same technique could be applied to
other kinds of random graphs, e.g., random <a
href="complex-networks.html">scale-free networks</a>?]
<li>Patrick Billinglsey, <cite>Convergence of Probability Measures</cite>
<li>Salomon Bochner, <cite>Harmonic Analysis and the Theory of Probability</cite>
<li>Tapas Kumar Chandra, <cite><a href="http://www.springer.com/book/978-81-322-0676-7">The Borel-Cantelli Lemma</a></cite>
<li>Louis H. Y. Chen, Larry Goldstein and Qi-Man Shao, <citE><a href="http://www.powells.com/partner/35751/biblio/9783642150067">Normal
Approximation by Stein's Method</a></cite>
<li>I. Calvo, J. C. Cuchí, J. G. Esteve, F. Falceto,
"Generalized Central Limit Theorem and Renormalization Group", <a href="http://arxiv.org/abs/1009.2899">arxiv:1009.2899</a>
<li>Sourav Chatterjee, "A new method of normal approximation",
<a href="http://arxiv.org/abs/math/0611213">arxiv:math/0611213</a>
<li>Bernard Chazelle, <Cite>The Discrepency Method: Randomness and
Complexity</cite>
<li>Irene Crimaldi and Luca Pratelli, "Two inequalities for conditional
expectations and convergence results for filters", <a
href="http://dx.doi.org/10.1016/j.spl.2005.04.039"><cite>Statistics and
Probability Letters</cite> <strong>74</strong> (2005): 151--162</a>
<li>Victor De La Pena and Evarist Gine, <cite>Decoupling: From
Dependence to Independence</cite>
<li>Victor H. de la Pena, Tze Leung Lai and Qi-Man Shan, <cite><a href="http://www.springer.com/math/probability/book/978-3-540-85635-1">Self-Normalized Processes: Limit Theory and Statistical Applications</a></cite>
<li>Janos Galambos and Italo Simonelli, <cite>Bonferroni-type
Inequalities with Applications</cite>
<li>Stefano Galatolo, Mathieu Hoyrup, Cristobal Rojas, "A constructive Borel-Cantelli Lemma. Constructing orbits with required statistical properties", <a href="http://arxiv.org/abs/0711.1478">arxiv:0711.1478</a>
<li>J. A. Gonzalez, L. I. Reyes, J. J. Suarez, L. E. Guerrero, and
G. Gutierrez, "A mechanism for randomness," <a
href="http://arxiv.org/abs/nlin.CD/0202022">nlin.CD/0202022</a> [Color me
skeptical, from the abstract]
<li>Martin Hairer, "A theory of regularity structures", <a href="http://arxiv.org/abs/1303.5113">arxiv:1303.5113</a>
<li>Oliver Johnson and Andrew Barron, "Fisher Information inequalities
and the Central Limit Theorem,"
<a href="http://arxiv.org/abs/math.PR/0111020">math.PR/0111020</a>
<li>Oliver Johnson and Richard Samworth, "Central Limit Theorem and
convergence to stable laws in Mallows distance", <a
href="http://arxiv.org/abs/math.PR/0406218">math.PR/0406218</a>
<li>Laurent Mazliak, "Poincarés Odds", <a href="http://arxiv.org/abs/1211.5737">arxiv:1211.5737</a>
<li>Henry McKean, <cite><a href="http://cambridge.org/9781107053212">Probability: The Classical Limit Theorems</a></cite>
<li>National Research Council (USA), <cite>Probability and
Algorithms</cite> [<a
href="http://www.nap.edu/books/0309047765/html/">online</a>]
<li>Peter Orbanz, "Projective limit random probabilities on Polish spaces", <a href="http://dx.doi.org/10.1214/11-EJS641"><cite>Electronic Journal of Statistics</cite>
<strong>5</strong> (2011): 1354--1373</a>
<li>Giovanni Peccati and Murad S. Taqqu, "Moments, cumulants and diagram formulae for non-linear functionals of random measures", <a href="http://arxiv.org/abs/0811/1726">arxiv:0811/1726</a>
<li>Iosif Pinelis, "Between Chebyshev and Cantelli", <a href="http://arxiv.org/abs/1011.6065">arxiv:1011.6065</a>
<li>Chris Preston, "Some notes on standard Borel and related
spaces", <a href="http://arxiv.org/abs/0809.3066">arxiv:0809.3066</a>
<li>Revesz, <cite>The Laws of Large Numbers</cite>
<li>R. Schweizer and A. Sklar, <cite>Probabilistic Metric Spaces</cite>
<li>Glenn Shafer and Vladimir Vovk, <cite>Probability and Finance: It's
Only a Game!</cite> [Yet Another Foundation of Probability, this time from
game-theory.]
<li>Akimichi Takemura, Vladimir Vovk, Glenn Shafer, "The generality of the zero-one laws", <a href="http://arxiv.org/abs/0803.3679">arxiv:0803.3679</a>
<li>Ramon van Handel, <a href="https://www.princeton.edu/~rvan/ORF570.pdf">Probability in High Dimension</a> [PDF lecture notes]
<li>Roman Vershynin, <cite><a href="http://cambridge.org/9781108415194">High-Dimensional Probability: An Introduction with Applications in Data Science </a></cite>
</ul>