Learning Gaussian mixture model with a maximization-maximization algorithm for image classification

The Expectation-Maximization (EM) algorithm is a well known iterative technique for learning a Gaussian Mixture Model (GMM). However, it is not without problems. (i) High computation cost when the number of Gaussians is huge. (ii) Heuristics are required to avoid singularities. (iii) Good initialization is required to avoid poor local maxima. In recent literature, Welling et al [1] suggested other alternatives to EM such as the Expectation-Expectation (EE), Maximization-Expectation (ME) and the Maximization-Maximization (MM) algorithm. Each of them share a trade-off between accuracy and speed. The MM algorithm is the fastest to compute while EE is the most accurate. In particular, the main difference between MM and EM lies in how the GMM latent variable is estimated. In this paper, we develop a new iterative technique for learning GMM based on the MM algorithm, known as the MM-GMM. Implementation only require computing a closed-form equation for GMM parameter and an optimization step for GMM latent variable. We tested our MM-GMM on two image categorization dataset and was able to outperform Kmeans when using vector quantization code such as BoW or VLAD for different cluster sizes.