Improved Generalization by Adding both Auto-Association and Hidden-Layer-Noise to Neural-Network-Based-Classifiers

We propose a novel method for learning that improves generalization in classifiers based on neural networks. The proposed method consists of (1) adding auto-associative learning and (2) simultaneously adding independent noise to the hidden layer of the neural-network. We verify this method with the classification problem of faces under variable illumination. Considering the interpolation for untrained samples as the key aspect of generalization, we expect that in our method, neural-classifiers will (1) learn (nearly) principal components of trained samples by auto-association, and will (2) generate and learn the variated samples from trained samples (along the axes of nearly principal components) by added noise, which leads both to increased amount of trained samples and (hopefully) to improved generalization.