### Mixture Models (Advanced Data Analysis from an Elementary Point of View)

From factor analysis to finite mixture models by allowing the latent variable
to be discrete. From kernel density estimation to mixture models by reducing
the number of points with copies of the kernel. Probabilistic formulation of
mixture models. Geometry: *q*+1 points define a *q*-dimensional
plane. Clustering. Estimation of mixture models by maximum likelihood, and
why it leads to a vicious circle. The expectation-maximization (EM,
Baum-Welch) algorithm replaces the vicious circle with iterative approximation.
More on the EM algorithm: convexity, Jensen's inequality, optimizing a lower
bound, proving that each step of EM increases the likelihood. Mixtures of
regressions. Other extensions.

PDF handout

Advanced Data Analysis from an Elementary Point of
View

Posted at April 09, 2011 23:50 | permanent link