Notebooks

## Tsallis Statistics, Statistical Mechanics for Non-extensive Systems and Long-Range Interactions

13 May 2021 10:28

A standard assumption of statistical mechanics is that quantities like energy are "extensive" variables, meaning that the total energy of the system is proportional to the system size; similarly the entropy is also supposed to be extensive. Generally, at least for the energy, this is justified by appealing to the short-range nature of the interactions which hold matter together, form chemical bonds, etc. But suppose one deals with long-range interactions, most prominently gravity; one can then find that energy is not extensive. This makes the life of the statistical mechanic much harder.

Constantino Tsallis is a physicist who came up with a supposed solution, based on the idea of maximum entropy. One popular way to derive the (canonical) equilibrium probability distribution is the following. One purports to know the average values of some quantities, such as the energy of the system, the number of molecules, the volume it occupies, etc. One then searches for the probability distribution which maximizes the entropy, subject to the constraint that it give the right average values for your supposed givens. Through the magic of Lagrange multipliers, the entropy-maximizing distribution can be shown to have the right, exponential, form, and the Lagrange multipliers which go along with your average-value constraints turn out to be the "intensive" variables paired with (or "conjugate to") the extensive ones whose means are constrained (energy <=> temperature, volume <=> pressure, molecular number <=> chemical potential, etc.). But, as I said, the entropy is an extensive quantity. What Tsallis proposed is to replace the usual (Gibbs) entropy with a new, non-extensive quantity, now commonly called the Tsallis entropy, and maximize that, subject to constraints. There is actually a whole infinite family of Tsallis entropies, indexed by a real-valued parameter q, which supposedly quantifies the degree of departure from extensivity (you get the usual entropy back again when q = 1). One can then grind through and show that many of the classical results of statistical mechanics can be translated into the new setting. What has really caused this framework to take off, however, is that while normal entropy-maximization gives you exponential, Boltzmann distributions, Tsallis statistics give you power-law, Pareto distributions, and everyone loves a power-law. (Strictly speaking, Tsallis distributions are type II generalized Pareto distributions, with power-law tails.) Today you'll find physicists applying Tsallis statistics to nearly anything with a heavy right tail.

I have to say I don't buy this at all. Leaving to one side my skepticism about the normal maximum entropy story, at least as it's usually told (e.g. by E. T. Jaynes), there are a number of features which make me deeply suspicious of Tsallis statistics.

1. It's simply not true that one maximizes the Tsallis entropy subject to constraints on the mean energy $\langle E \rangle =\sum_{i}{p_i E_i}$. Rather, to get things to work out, you have to fix the value of a "generalized mean" energy, ${ \langle E \rangle }_{q} = \sum_{i}{p_i^q E_i} / \sum_{i}{p^q_i}$. (This can be interpreted as replacing the usual average, an expectation take with respect to the actual probability distribution, by an expectation taken with respect to a new, "escort" probability distribution.) I have yet to encounter anyone who can explain why such generalized averages should be either physically or probabilistically natural; the usual answer I get is "OK, yes, it's weird, but it works, doesn't it?"
2. There is no information-theoretic justification for the Tsallis entropy, unlike the usual Gibbs entropy. The Tsallis form is, however, a kind of low-order truncation of the Rényi entropy, which does have information-theoretic interest. (The Tsallis form has been independently rediscovered many times in the literature, going back to the 1960s, usually starting from the Rényi entropy. A brief review of the "labyrinthic history of the entropies" can be found in one of Tsallis's papers, cond-mat/0010150.) Maximizing the Rényi entropy under mean-value constraints leads to different distributions than maximizing the Tsallis entropy.
3. I have pretty severe doubts about the backing story here, about long-range interactions leading to a non-extensive form for the entropy, particularly when, in derivations which begin with such a story, I often see people blithely factoring the probability that a system is in some global state into the product of the probabilities that its components are in various states, i.e., assuming independent sub-systems.
4. There are alternative, non-max-ent derivations of the usual statistical-mechanical distributions; such derivations do not seem forthcoming for Tsallis statistic. In particular, large deviations arguments, which essentially show how to get such distributions as emergent, probabilistic consequences of individual-level interactions, do not seem to ever lead to Tsallis statistics, even when one has the kind of long-range interactions which, supposedly, Tsallis statistics ought to handle.
5. There is no empirical evidence that Tsallis statistics correctly gives the microscopic energy distribution for any known system.
6. Zanette and Montemurro have shown that you can get any distribution you like out of the Tsallis recipe, simply by changing the function whose generalized average you take as your given. The usual power-law prescription only holds if you constrain either x or x2, but one of the more "successful" applications requires constraining the generalized mean of $x^{2\alpha}/2 - c\mathrm{sgn}{x}({|x|}^{\alpha} - {|x|}^{3\alpha}/3)$, with c and $\alpha$ as adjustable parameters! (In fairness, I should point out that if you're willing to impose sufficiently weird constraints, you can generate arbitrary distributions from the usual max. ent. procedure, too; this is one of the reasons why I don't put much faith in that procedure.)

I think the extraordinary success of what is, in the end, a slightly dodgy recipe for generating power-laws illustrates some important aspects, indeed unfortunate weaknesses, in the social and intellectual organization of "the sciences of complexity". But that rant will have to wait for my book on The Genealogy of Complexity, which, prudently, means waiting until I'm safely tenured. (Update, 2021: I am indeed now safely tenured, but I have better, or at least more pressing, things to do, so enjoy.)

I should also discuss the "superstatistics" approach here, which tries to generate non-Boltzmann statistics as mixtures of Boltzmann distributions, physically justified by appealing to fluctuating intensive variables, such as temperature. I will only remark that the superstatistics approach severes all connections between the use of these distributions and non-extensivity and long-range interactions; and that results in the statistical literature on getting generalized Pareto distributions from mixtures of exponentials go back to 1952 at least.

Finally, it has come to my attention that some people are citing this notebook as though it had some claim to authority. Fond though I am of my own opinions, this seems to me to be deeply wrong. The validity of Tsallis statistics, as a scientific theory, ought to be settled in the usual way, by means of the peer-reviewed scientific literature, subject to all its usual conventions and controls. It's obvious from the foregoing that I have pretty strong beliefs in how that debate ought to go, and (this may not be so clear) enough faith in the scientific community that I think, in the long run, it will go that way, but no one should confuse my opinion with a scientific finding. For myself, this page is a way to organize my own thoughts; for everyone else, it's either entertainment, or at best an opinionated collection of pointers to the real discussion.

Recommended, big picture:
• Tsallis & co. maintain a pretty comprehensive and ever-growing bibliography on Tsallis statistics. This includes replies to many of the papers I list here.
• Julien Barré, Freddy Bouchet, Thierry Dauxois and Stefano Ruffo, "Large deviation techniques applied to systems with long-range interactions", cond-mat/0406358 = Journal of Statistics Physics 119 (2005): 677--713 [What large deviation results for long-range interactions look like]
• Christian Beck, "Superstatistics: Recent developments and applications", cond-mat/0502306
• Freddy Bouchet and Thierry Dauxois, "Prediction of anomalous diffusion and algebraic relaxations for long-range interacting systems, using classical statistical mechanics", Physical Review E 72 (2005): 045103 = cond-mat/0407703
• Thierry Dauxois, "Non-Gaussian distributions under scrutiny", Journal of Statistical Mechanics (2007) N08001
• Brian R. La Cour and William C. Schieve, "A Comment on the Tsallis Maximum Entropy Principle", Physical Review E 62 (2000): 7494--7496, cond-mat/0009216
• Michael Nauenberg, "Critique of q-entropy for thermal statistics", Physical Review E 67 (2003): 036114 [From the abstract: "[I]t is shown here that the joint entropy for systems having different values of q is not defined in this formalism, and consequently fundamental thermodynamic concepts such as temperature and heat exchange cannot be considered for such systems. Moreover, for q &neq; 1 the probability distribution for weakly interacting systems does not factor into the product of the probability distribution for the separate systems, leading to spurious correlations and other unphysical consequences, e.g., nonextensive energy, that have been ignored in various applications given in the literature." That the probabilities for sub-systems do not factor is, I think, especially devastating, because almost all of the work on the subject assumes that it does. See also comment by Tsallis, cond-mat/305091, and reply by Nauenberg, cond-mat/0305365, which I believe to be correct.]
• Hugo Touchette, "Comment on 'Towards a large deviation theory for strongly correlated systems' ", arxiv:1209.2611
• Damién H. Zanette and Marcelo A. Montemurro
• "A note on non-therrmodynamical applications of non-extensive statistics", cond-mat/0305070 = Physics Letters A 324 (2004): 383--387 [An amusing and quite conclusive assault, culminating in a demonstration that you can use the non-extensive formalism to "derive" any probability distribution whatsoever.]
• "Thermal measurement of stationary nonequilibrium systems: A test for generalized thermostatistics", Physics Letters A 316 (2003): 184--189 = cond-mat/0212327 [And it doesn't even work for for thermodynamic systems.]
Recommended, close-ups:
• Freddy Bouchet, Thierry Dauxois, Stefano Ruffo, "Controversy about the applicability of Tsallis statistics to the HMF model", cond-mat/0605445 = Europhysics News 37 (2006): 9--10
• G. Baris Bagci, Thomas Oikonomou, "Do Tsallis distributions really originate from the finite baths?", arxiv:1305.2493
• A. G. Bashkirov, "Comment on 'Stability of Tsallis entropy and instabilities of Rényi and normalized Tsallis entropies: A basis for q-exponential distributions'," Physical Review E 72 (2005): 028101 [There is also a reply by S. Abe, the author of the original article, which, predictably, I find unconvincing: Physical Review E 72 (2005): 028102.]
• Alice M. Crawford, Nicolas Mordant, Andy M. Reynolds, Eberhard Bodenschatz, "Comment on 'Dynamical Foundations of Nonextensive Statistical Mechanics'", physics/0212080
• Peter Grassberger
• "Temporal scaling at Feigenbaum points and non-extensive thermodynamics", cond-mat/0508110 = Physical Review Letters 95 (2005): 140601 [I can't resist quoting the abstract in full, if only because I enjoy Prof. Grassberger's no-quarter-asked-or-given tone: "We show that recent claims for the non-stationary behaviour of the logistic map at the Feigenbaum point based on non-extensive thermodynamics are either wrong or can be easily deduced from well-known properties of the Feigenbaum attractor. In particular, there is no generalized Pesin identity for this system, the existing 'proofs' being based on misconceptions about basic notions of ergodic theory. In deriving several new scaling laws of the Feigenbaum attractor, thorough use is made of its detailed structure, but there is no obvious connection to non-extensive thermodynamics." One point made here (but passed over in the abstract) is that there are nearly as many estimates of the "right" value of the non-extensivity parameter q at the period-doubling accumulation point as there are papers on the system. This tends to reduce one's confidence that any of them is a physically meaningful parameter.]
• "Proposed central limit behavior in deterministic dynamical systems", Physical Review E 79 (2009): 057201<
• H. J. Hilhorst
• "Central limit theorems for correlated variables: some critical remarks", Brazilian Journal of Physics 39 (2000): 371--379, arxiv:0901.1249
• "Note on a q-modified central limit theorem", Journal of Statistical Mechanics (2010): P10023, arxiv:1008.4259
• H. J. Hilhorst and G. Schehr, "A note on q-Gaussians and non-Gaussians in statistical mechanics", Journal of Statistical Mechanics (2007) P06003 [Analytical results on the limiting distributions of certain sums of correlated random variables, supposed to follow "q-Gaussians", but not actually doing so. It strikes me as extraordinary that no one in this literature, on either side, pays any attention to actual results in probability theory about generalizations of the central limit theorem; one searches these bibliographies in vain for names like Lévy and Rosenblatt.]
• B. H. Lavenda and J. Dunning-Davies, "Additive Entropies of degree-q and the Tsallis Entropy", physics/0310117
Modesty forbids me to recommend:
• CRS, "Maximum Likelihood Estimation for q-Exponential (Tsallis) Distributions", math.ST/0701854 [If you have to use these things, you really should estimate their parameters this way, and not try to fit curves to the sample distribution function.]
• CRS and Alessandro Rinaldo, "Consistency under Sampling of Exponential Random Graph Models", Annals of Statistics 41 (2013): 508--535, arxiv:1111.3054 [While we didn't emphasize it in the paper, considering our audience and primary application, one way to understand this is as a study of the weirdness that can result within a canonical ensemble where interactions are not short range, so that the total energy of a system divided into two large sub-systems doesn't break down into the internal, volume-proportional energy of each sub-system, plus a comparatively small term strictly from the boundary.]
• Andrea Antoniazzi, Francesco Califano, Duccio Fanelli, and Stefano Ruffo, "Exploring the Thermodynamic Limit of Hamiltonian Models: Convergence to the Vlasov Equation", Physical Review Letters 98 (2007): 150602
• R. Bachelard, C. Chandre, D. Fanelli, X. Leoncini and S. Ruffo, "Abundance of Regular Orbits and Nonequilibrium Phase Transitions in the Thermodynamic Limit for Long-Range Systems", Physical Review Letters 101 (2008): 260603
• Fulvio Baldovin, Pierre-Henri Chavanis and Enzo Orlandini, "Microcanonical quasistationarity of long-range interacting systems in contact with a heat bath", Physical Review E 79 (2009): 011102
• Fulvio Baldovin and Enzo Orlandini
• "Hamiltonian Dynamics Reveals the Existence of Quasistationary States for Long-Range Systems in Contact with a Reservoir", Physical Review Letters 96 (2006): 240602 = cond-mat/0603383 ["We introduce a Hamiltonian dynamics for the description of long-range interacting systems in contact with a thermal bath (i.e., in the canonical ensemble). The dynamics confirms statistical mechanics equilibrium predictions for the Hamiltonian mean field model and the equilibrium ensemble equivalence. We find that long-lasting quasistationary states persist in the presence of the interaction with the environment. Our results indicate that quasistationary states are indeed reproducible in real physical experiments."]
• "Quasi-stationary states in long-range interacting systems are incomplete equilibrium states", cond-mat/0603659 = Physical Review Letters 97 (2006): 100601 ["Despite the presence of an anomalous single-particle velocity distribution, we find that ordinary Central Limit Theorem leads to the Boltzmann factor in Gibbs' $\Gamma$-space. We identify the non-equilibrium sub-manifold of $\Gamma$-space responsible for the anomalous behavior and show that by restricting the Boltzmann-Gibbs approach to such sub-manifold we obtain the statistical mechanics of the quasi-stationary states."]
• Christian Beck, "Generalized information and entropy measures in physics", arxiv:0902.1235
• Christian Beck, Ezechiel G. D. Cohen and Harry L. Swinney, "From time series to superstatistics", Physical Review E 72 (2005): 056133
• Freddy Bouchet, "Stochastic process of equilibrium fluctuations of a system with long-range interactions", Physical Review E 70 (2004): 036113
• F. Bouchet and J. Barré, "Classification of Phase Transitions and Ensemble Inequivalence, in Systems with Long Range Interactions", Journal of Statistical Physics 118 (2005): 1073--1105
• Pierre-Henri Chavanis
• "Dynamics and thermodynamics of systems with long-range interactions: interpretation of the different functionals", arxiv:0904.2729
• "Statistical mechanics of geophysical turbulence: application to jovian flows and Jupiter's great red spot", Physica D 200 (2005): 257--272 [Listed here because this is (judging by the abstract) an instance of Chavanis's more general non-Tsallisite (non-Tsallisian?) approach to statistical mechanics with long-range interactions]
• "Generalized Fokker-Planck equations and effective thermodynamics", cond-mat/0504716 = Physica A 340 (2004): 57
• "Quasi-stationary states and incomplete violent relaxation in systems with long-range interactions", cond-mat/0509726
• "Lynden-Bell and Tsallis distributions for the HMF model", cond-mat/0604234
• Pierre-Henri Chavanis, C. Rosier and C. Sire, "Thermodynamics of self-gravitating systems," cond-mat/0107345
• Thierry Dauxois, Stefano Ruffo, Ennio Arimondo and Martin Wilkens (eds.), Dynamics and Thermodynamics of Systems With Long Range Interactions
• Davide Ferrari and Yuhong Yang, "Maximum Lq-likelihood estimation", Annals of Statistics 38 (2010): 753--783
• V. Garcia-Morales, J. Pellicer, "Statistical mechanics and thermodynamics of complex systems", math-ph/0304013
• Toshiyuki Gotoh, Robert H. Kraichnan, "Turbulence and Tsallis Statistics", nlin.CD/0305040
• D. H. E. Gross, "Non-extensive Hamiltonian systems follow Boltzmann's principle not Tsallis statistics," cond-mat/0106496
• Shamik Gupta, David Mukamel, "Slow relaxation in long-range interacting systems with stochastic dynamics", arxiv:1006.0233
• Rudolf Hanel and Stefan Thurner, "On the Derivation of power-law distributions within standard statistical mechanics", cond-mat/0412016 = Physica A 351 (2005): 260--268
• Petr Jizba and Toshihico Arimitsu, "The world according to Rényi: Thermodynamics of multifractal systems," cond-mat/0207707
• Ramandeep S. Johal, Antoni Planes, and Eduard Vives, "Equivalence of nonadditive entropies and nonadditive energies in long range interacting systems under macroscopic equilibrium", cond-mat/0503329
• T. Kodama, H.-T. Elze, C. E. Aguiar, T. Koide, "Dynamical Correlations as Origin of Nonextensive Entropy", cond-mat/0406732
• Hiroko Koyama, Tetsuro Konishi, and Stefano Ruffo, "Clusters die hard: Time-correlated excitation in the Hamiltonian Mean Field model", nlin.CD/0606041
• Bernard H. Lavenda
• "Fundamental inconsistencies of 'superstatistics'", cond-mat/0408485
• "Information and coding discrimination of pseudo-additive entropies (PAE)", cond-mat/0403591
• Massimo Marino, "Power-law distributions and equilibrium thermodynamics", cond-mat/0605644 [Makes the interesting claims that if you want a consistent thermodynamics with power-law distributions, then the entropy is uniquely determined to be the Rényi entropy, not the Tsallis entropy]
• David Mukamel
• "Statistical Mechanics of systems with long range interactions", arxiv:0811.3120
• "Notes on the Statistical Mechanics of Systems with Long-Range Interactions", arxiv:0905.1457
• D. Mukamel, S. Ruffo and N. Schreiber, "Breaking of ergodicity and long relaxation times in systems with long-range interactions", cond-mat/0508604
• Jan Naudts, "Parameter estimation in nonextensive thermostatistics", cond-mat/0509796
• Ouassim Feliachi, Freddy Bouchet, "Dynamical large deviations for homogeneous systems with long range interactions and the Balescu--Guernsey--Lenard equation", arxiv:2105.05644
• A. S. Parvana and T.S. Biró, "Extensive Rényi statistics from non-extensive entropy", Physics Letters A 340 (2005): 375--387
• Aurelio Patelli, Shamik Gupta, Cesare Nardini, and Stefano Ruffo, "Linear response theory for long-range interacting systems in quasistationary states", Physical Review E 85 (2012): 021133
• Daniel Pfenniger, "Virial statistical description of non-extensive hierarchical systems", cond-mat/0605665
• Alessandro Pluchino, Vito Latora and Andrea Rapisarda, "Dynamics and Thermodynamics of a model with long-range interactions", cond-mat/0410213
• S. M. Duarte Queirós and C. Tsallis, "Bridging a paradigmatic financial model and nonextensive entropy", Europhysics Letters 69 (2005): 893--899 [Approximation of ARCH model using Tsallis entropies. Thanks to Nick Watkins for bringing this to my attention.]
• M. S. Reis, V. S. Amaral, R. S. Sarthour and I. S. Oliveira, "Experimental determination of the non-extensive entropic parameter $q$", cond-mat/0512208
• T. M. Rocha Filho, M. A. Amato, B. A. Mello, and A. Figueiredo, "Phase transitions in simplified models with long-range interactions", Physical Review E 84 (2011): 041121
• T. M. Rocha Filho, A. Figueiredo, and M. A. Amato, "Entropy of Classical Systems with Long-Range Interactions", Physical Review Letters 95 (2005): 190601
• Stefano Ruffo, "Equilibrium and nonequilibrium properties of systems with long-range interactions", European Physical Journal B 64 (2008): 355--363, arxiv:0711/1173

Previous versions: 27 Feb 2017 16:30; 2007-01-29 23:22; first version several years older (2003? earlier?)