A theory of over-learning in the presence of noise

The over-learning problem for multilayer feedforward neural networks is discussed. A framework is proposed for the over-learning problem with noise free training data. It is shown that the framework is still valid in the case of noisy training data. It is applied to the case where the rote memorization criterion is used as a substitute for the Wiener criterion. Necessary and sufficient conditions for two kinds of admissibility of the rote memorization criterion by the Wiener criterion are obtained. These conditions lead to a method for choosing a training set which prevents Wiener-over-learning.<>