A Gentle Tutorial of the EM Algorithm

A Gentle Tutorial of the EM Algorithm and its Applicationto Parameter Estimation for GaussianMixture and Hidden Markov Models

We describe the
maximum-likelihood parameter estimation problem and how the
Expectation-Maximization (EM) algorithm can be used for its solution. We
first describe the abstract form of the EM algorithm as it is often given
in the literature. We then develop the EM parameter estimation procedure
for two applications: 1) finding the parameters of a mixture of Gaussian
densities, and 2) finding the parameters of a hidden Markov model (HMM)
(i.e.,the Baum-Welch algorithm) for both discrete and Gaussian mixture
observation models.We derive the update equations in fairly explicit
detail but we do not prove any convergence properties. We try to emphasize
intuition rather than mathematical rigor.

download