### Mixture Models (Advanced Data Analysis from an Elementary Point of View)

From factor analysis to mixture models by allowing the latent variable to
be discrete. From kernel density estimation to mixture models by reducing the
number of points with copies of the kernel. Probabilistic formulation of
mixture models. Geometry: planes again. Probabilistic clustering. Estimation
of mixture models by maximum likelihood, and why it leads to a vicious circle.
The expectation-maximization (EM, Baum-Welch) algorithm replaces the vicious
circle with iterative approximation. More on the EM algorithm: convexity,
Jensen's inequality, optimizing a lower bound, proving that each step of EM
increases the likelihood. Mixtures of regressions. Other extensions.

Extended example: Precipitation in Snoqualmie Falls revisited. Fitting a
two-component Gaussian mixture; examining the fitted distribution; checking
calibration. Using cross-validation to select the number of components to use.
Examination of the selected mixture model. Suspicious patterns in the
parameters of the selected model. Approximating complicated distributions
vs. revealing hidden structure. Using bootstrap hypothesis testing to select
the number of mixture components.

*Reading*:
Notes, chapter
20; `mixture-examples.R`

Advanced Data Analysis from an Elementary Point of View

Posted at April 15, 2012 20:00 | permanent link