Improving generalization by using genetic algorithms to determine the neural network size

Recent theoretical results support that decreasing the number of free parameters in a neural network (i.e., weights) can improve generalization. The importance of these results has triggered the development of many approaches which try to determine an "appropriate" network size for a given problem. Although it has been demonstrated that most of the approaches manage to find small size networks which solve the problem at hand, it is quite remarkable that the generalization capabilities of these networks have not been explored thoroughly. In this paper, we propose the coupling of genetic algorithms and weight pruning with the objective of both reducing network size and improving generalization. The innovation of our approach relies on the use of a fitness function which uses an adaptive parameter to encourage the reproduction of networks having good generalization performance and a relatively small size.