Maximum likelihood density estimation by means of a PDP network

Based on the principle of maximizing the likelihood of proper classification of training samples, an algorithm is proposed to train the artificial neural pattern density estimator (parallel distributed processing (PDP) network) introduced by the authors earlier (1990). The previous restrictions on unit functions were relaxed such that each unit in the network represented a joint density of independent Gaussian variables with equal variances while variances across densities did not have to be the same. The algorithm was tested with samples derived from known mixtures of memoryless Gaussian sources as well as exponential and Gamma densities. Both one- and two-dimensional cases were explored. The success of the network in estimating the probability density functions depended on how well they were represented by the training samples, the number of hidden units employed and how thoroughly the network was trained. The results of comparing the network's recognition rates against those of a Bayes classifier are presented.<>