A constructive EM approach to density estimation for learning

Density estimation based on a mixture of Gaussian components is particularly suited to the solution of function approximation problems. When dealing with numerical examples of the function to be approximated, the corresponding neural network architecture can be trained by using a clustering procedure based on the well-known EM algorithm. However, the latter is characterized by some serious drawbacks that we overcome in this paper. For we propose a constructive procedure that increases progressively the number of Gaussian components; it yields improvements of both the speed and the quality of the EM convergence. Moreover, it also drastically reduces the computational cost of the optimization procedure that we further propose in order to select automatically the optimal number of Gaussian components of the neural network. The performance of the proposed approach is compared in the paper with respect to well-known neural network approaches.