Improvement of EM algorithm by means of non-extensive statistical mechanics

We propose a new type of EM algorithm based on Tsallis non-extensive statistical mechanics. In our algorithm, the posterior distribution is derived in terms of the principle of maximizing Tsallis generalized entropy. Then, the problem is reformulated so as to minimize a generalized free energy in order to maximize a incomplete data log-likelihood function indirectly. We control a parameter q, which represents non-extensive property of entropy, for each EM step so that q goes to 1 at final stage of the algorithm. In order to check the efficiency of our method, the algorithm is applied to the Gaussian mixture means estimation problems. We find that the results of our algorithm are better than those of the conventional EM algorithm or the DAEM algorithm.