Invariance constraints for improving generalization in probabilistic neural networks

Probabilistic neural networks can approximate class conditional densities in optimal (Bayesian) pattern classifiers. In natural pattern recognition applications, the size of the training set is always limited, making the approximation task difficult. Invariance constraints can significantly simplify the task of density approximation. A technique is presented for learning invariant representations, based on a statistical approach to ground invariance. An iterative method is developed formally for computing the maximum likelihood estimate to the parameters of an invariant mixture model. The method can be interpreted as a competitive training strategy for a radial basis function (RBF) network. It can be used for self-organizing formation of both invariant templates and features.<>