Inter-iteration optimization of parallel EM algorithm on message-passing multicomputers

Estimation of the parameters of a probability distribution function is a complicated problem that is frequently encountered in many instances of real world problems. The Expectation Maximization (EM) algorithm often can be employed when there is a many-to-one mapping from all possible distribution patterns to the distribution governing the outcome. With its maximum likelihood (ML) formulation, optimal estimate can be made for the unknown variables after iterations until convergence. A variety of parallel methods have been proposed to boost its performance because of the complexity involved in the algorithm. Despite the efforts, the ML algorithm could not be easily adopted in practice primarily due to both intra- and inter-iteration data dependence problems resulting from the iterative nature of the algorithm. This research builds upon experimentation that demonstrated promising results in speeding up the algorithm in and between iterations using distributed memory message passing architecture.