Optimizing multilayer perceptrons by discriminatory component analysis

Multilayer perceptrons offer an integrated procedure for feature extraction and Bayes classification by learning the decision boundary. Its feedforward autoassociative architecture can also be used to construct subspaces in a supervised or unsupervised model [A.K. Jain et al., 2000]. On the other hand, multiclass linear discriminant analysis provides a multivariate prediction by estimating the density function. Its linear subspaces obtained by the weighted Fisher criteria under a standard finite normal mixture model retain most closely the intrinsic Bayes separability [M. Loog et al., 2001]. Here we show a two-fold connection between multilayer perceptrons and linear discriminant analysis using discriminatory component analysis. This theoretical observation immediately suggests a possible clustering-model supported optimization mechanism for multilayer perceptrons: the weights between the input and hidden layers are related to eigenvectors of the weighted Fisher scatter matrix, the number of the hidden layer neurons is justified by the corresponding significant eigenvalues, and the weights connected to the output neurons are obtained from the centers of the classes in the extracted feature subspaces