Averaging ensembles of self-organising mixture networks for density estimation

The self-organising mixture network (SOMN) is a learning algorithm for mixture densities, derived from minimising the Kullback-Leibler information by means of stochastic approximation methods. It has been shown the SOMN converges faster than the EM-based algorithms and generalises better as it is based on the expected likelihood rather than the sample likelihood. The derived algorithm has similar updating forms to the self-organising map (SOM), thus reveals the mixture interpreter role of the neighbourhood function used in the SOM. When the sample set is small, overfitting problems often occur in most algorithms. Further improvement can be achieved by averaging ensembles of the SOMNs. The algorithms have been applied to both experimental data and real-world problems. The results show that smoothed mixtures with improved accuracy have been obtained. Estimation variance has been reduced.