April 09, 2011

Mixture Models (Advanced Data Analysis from an Elementary Point of View)

From factor analysis to finite mixture models by allowing the latent variable to be discrete. From kernel density estimation to mixture models by reducing the number of points with copies of the kernel. Probabilistic formulation of mixture models. Geometry: q+1 points define a q-dimensional plane. Clustering. Estimation of mixture models by maximum likelihood, and why it leads to a vicious circle. The expectation-maximization (EM, Baum-Welch) algorithm replaces the vicious circle with iterative approximation. More on the EM algorithm: convexity, Jensen's inequality, optimizing a lower bound, proving that each step of EM increases the likelihood. Mixtures of regressions. Other extensions.

PDF handout

Advanced Data Analysis from an Elementary Point of View

Posted at April 09, 2011 23:50 | permanent link

Three-Toed Sloth