## Mixture Models (in Statistics)

*19 Apr 2015 14:11*

Yet Another Inadequate Placeholder.

See also: Clustering; Density Estimation; Ensemble Methods in Machine Learning; Factor Models; Model Selection; Statistical Inference in Markov and Hidden Markov Models; Statistics

- Recommended, big picture:
- David J. Bartholomew, Latent Variable Models and Factor Analysis
- Hastie, Tibshirani, and Friedman, The Elements of Statistical Learning

- Recommended, close ups:
- Animashree Anandkumar, Daniel Hsu, Sham M. Kakade, "A Method of Moments for Mixture Models and Hidden Markov Models", arxiv:1203.0683
- David M. Blei, Andrew Y. Ng and Michael I. Jordan, "Latent Dirichlet Allocation", Journal of Machine Learning Research
**3**(2003): 993--1022 - Jiahua Chen and Xianming Tan, "Inference for Multivariate Normal
Mixtures", arxiv:0805.3906 [Little
of the machinery looks like it depends on using mixtures of
*normals*] - Jonathan K. Pritchard, Matthew Stephens and Peter Donnelly,
"Inference of Population Structure Using Multilocus Genotype
Data", Genetics
**155**(2000): 945--959 [As "arthegall" pointed out on Pinboard (i, ii), this is the same model as in the famous Blei/Ng/Jordan paper, but applied to genes, rather than words.] - Babak Shahbaba, "Discovering Hidden Structure Using Mixture Models: Application to Nonlinear Time Series Processes", Studies in Nonlinear Dynamics and Econometrics
**13:2**(2009): 5 [Not as groundbreaking as it makes itself sound, but still interesting]

- Modesty forbids me to recommend:
- The chapter on mixture models in Advanced Data Analysis from an Elementary Point of View

- To read:
- Elizabeth S. Allman, Catherine Matias, John A. Rhodes, "Identifiability of parameters in latent structure models with many observed variables",
Annals of Statistics
**37**(2009): 3099--3132, arxiv:0809.5032 - A. Anandkumar, D. Hsu, S. M. Kakade, "Learning High-Dimensional Mixtures of Graphical Models", arxiv:1203.0697
- Florentina Bunea, Alexandre B. Tsybakov, Marten H. Wegkamp, Adrian Barbu, "Spades and Mixture Models", Annals of Statistics
**38**(2010): 2525--2558, arxiv:0901.2044 - Olivier Cappé, Eric Moulines, "On-line expectation-maximization algorithm for latent data models", Journal of the Royal Statistical
Society B
**71**(2009): 593--613, arxiv:0712.4273 - Gilles Celeux, Stéphane Chrétien, Florence Forbes, "A Component-wise EM Algorithm for Mixtures", Journal of Computational and Graphical Statistics
**10**(2001): 697--712, arxiv:1201.5913 - Elisabeth Gassiat, Ramon Van Handel
- "The local geometry of finite mixtures", arxiv:1202.3482
- "Consistent order estimation and minimal penalties", arxiv:1002.1280

- Lancelot F. James, David J. Marchette and Carey Priebe, "Consistent
estimation of mixture
complexity", Annals of
Statistics
**29**(2001): 1281--1296 - Hiroyuki Kasahara and Katsumi Shimotsu, "Non-parametric identification and estimation of the number of components in multivariate mixtures", Journal of the Royal Statistical Society: Series B forthcoming
- Mikhail Kovtun, Igor Akushevich, Kenneth G. Manton and H. Dennis
Tolley
- "Linear Latent Structure Analysis: Mixture Distribution Models with Linear Constraints", math.PR/0507025
- "A New Efficient Algorithm for Construction of LLS Models", math.PR/0507021

- Yanyuan Ma, Jeffrey D. Hart and Raymond J. Carroll, "Density Estimation in Several Populations With Uncertain Population Membership", Journal of the American Statistical Association
**106**(2011): 1180--1192 - Geoffrey J. McLachlan and David Peel, Finite Mixture Models
- Volodymyr Melnykov and Ranjan Maitra, "Finite mixture models and model-based clustering", Statistics Surveys
**4**(2010): 80--116 - Frank Nielsen, "$k$-MLE: A fast algorithm for learning statistical mixture models", arxiv:1203.5181
- M. Pavlic and M. J. van der Laan, "Fitting of mixtures with
unspecified number of components using cross validation distance
estimate", Computational Statistics and Data
Analysis
**41**(2003): 413--428 - Titterington
- Guenther Walther, "The Average Likelihood Ratio for Large-scale Multiple Testing and Detecting Sparse Mixtures", arxiv:1111.0328