Random Matrix Theory

02 Oct 2024 11:30

---

Yet Another Inadequate Placeholder

--- I've decided that I finally need to learn this, because it seems relevant to two separate projects. One of them could be expressed as follows: if I have absolutely garbage data, with no low-dimensional structure at all. but I nonetheless subject it to principal components analysis, how much of the variance can I expect the first few PCs to "explain"? If I similarly subject garbage data to factor analysis, how well can I expect factor models to seem to fit? What about if the garbage data is restricted to have all-non-negative correlations across variables?

The other project is more constructive, but also more complicated than I feel like explaining here.

See also:
• Concentration of Measure
• Factor Models
• Large Deviations
• Math I Ought to Learn
• Random Feature Methods in Machine Learning
• Stability and Complexity of Ecosystems
• Thomson Sampling Model
• What Is the Right Null Model for Linear Regression?

Recommended, big picture (for what this is worth, given my confessed newbieness):
• Marc Potters and Jean-Philippe Bouchaud, A First Course in Random Matrix Theory: for Physicists, Engineers and Data Scientists

Recommended, close-ups (ditto):
• Philipp Fleig and Ilya Nemenman, "Statistical properties of large data sets with linear latent features", Physical Review E 106 (2022): 014102, arxiv:2111.04641 [Comments]

To read, big pictures:
• Greg W. Anderson, Alice Guionnet and Ofer Zeitouni, An Introduction to Random Matrices
• Gordon Blower, Random Matrices: High Dimensional Phenomena
• Arup Bose, Patterned Random Matrices
• Romain Couillet and Zhenyu Liao, Random Matrix Methods for Machine Learning
• Alan Julian Izenman, "Random Matrix Theory and Its Applications", Statistical Science 36 (201) 421--442
• Vladislav Kargin, Elena Yudovina, "Lecture Notes on Random Matrix Theory", arxiv:1305.2153
• Elizabeth Meckes, "The Eigenvalues of Random Matrices", IMAGE, the Bulletin of the International Linear Algebra Society 65 (2020); 9--22, arxiv:2101.02928
• Terry Tao, Topics in Random Matrix Theory

To read, close-ups:
• Arash A. Amini and Zahra S. Razaee, "Concentration of kernel matrices with application to kernel spectral clustering", Annals of Statistics 49 (2021): 531--556
• Zhigang Bao, Liang-Ching Lin, Guangming Pan, Wang Zhou, "Spectral statistics of large dimensional Spearman's rank correlation matrix and its application", arxiv:1312.5119
• F. Benaych-Georges, A. Guionnet, and M. Maida, "Large deviations of the extreme eigenvalues of random deformations of matrices", Probability Theory and Related Fields 154 (2012): 703--751
• Giulio Biroli, Alice Guionnet, "Large deviations for the largest eigenvalues and eigenvectors of spiked random matrices", arxiv:1904.01820
• Alex Bloemendal, Antti Knowles, Horng-Tzer Yau, Jun Yin, "On the principal components of sample covariance matrices", arxiv:1404.0788
• S. G. Bobkov and F. Götze, "Concentration of empirical distribution functions with applications to non-i.i.d. models", Bernoulli 16 (2010): 1385--1414, arxiv:1011.6165
• Marco Chiani, "Distribution of the largest eigenvalue for real Wishart and Gaussian random matrices and a simple approximation for the Tracy-Widom distribution", Journal of Multivariate Analysis 129 (2014): 69--81, arxiv:1209.3394
• Philipp Fleig, Ilya Nemenman, "Generative probabilistic matrix model of data with different low-dimensional linear latent structures", arxiv:2212.02987
• Alice Guionnet, "Large deviations and stochastic calculus for large random matrices", Probability Surveys 1 (2004): 72--172
• Johanna Hardin, Stephan Ramon Garcia, and David Golan, "A method for generating realistic correlation matrices", Annals of Applied Statistics 7 (2013): 1733--1762, arxiv:1106.5834
• Alex James, Michael J. Plank, Axel G. Rossberg, Jonathan Beecham, Mark Emmerson, Jonathan W. Pitchford, "Constructing Random Matrices to Represent Real Ecosystems", American Naturalist 185 (2015): 680--692
• Steven Soojin Kim and Kavita Ramanan, "Large deviation principles induced by the Stiefel manifold, and random multi-dimensional projections", arxiv:2105.04685
• Satya N. Majumdar and Pierpaolo Vivo, "Number of Relevant Directions in Principal Component Analysis and Wishart Random Matrices", Physical Review Letters 108 (2012): 200601
• Faheem Mosam, Diego Vidaurre, Eric De Giuli, "Breakdown of random matrix universality in Markov models", Physical Review E 104 (2021): 024305, arxiv:2105.04393
• Natesh S. Pillai, and Jun Yin, "Universality of covariance matrices", Annals of Applied Probability 24 (2014): 935--1001, arxiv:1110.2501
• Joel A. Tropp, "An Introduction to Matrix Concentration Inequalities", arxiv:1501.01571
• Van Vu, Ke Wang, "Random weighted projections, random quadratic forms and random eigenvectors", arxiv:1306.3099