Optimising Artificial Neural Network topologies using Genetic Algorithms with very small populations

In this paper, we show that searching for good Artificial Neural Network (ANN) topologies can be done by Genetic Algorithms (GA) using very small populations, as low as 2 individuals. We use an Island Model Memetic Algorithm, where evolution optimises the type, number and size of layers, and the weights are adjusted through Backpropagation. We test this algorithm on the MNIST dataset. Although we start on population sizes as large as 125 (5 islands of 25 individuals each), and obtain the best results (96%-97% accuracy) on a population size of 100 (10 islands of 10 individuals each), GAs with population sizes as low as 2 use 680 fitness evaluations to produce ANNs reaching, in the best case, 96%-97% accuracy. The main drawback is that low population sizes induce result instability: in the average case, the accuracy is between 92% and 95%. We explain the results, in part, as due to representing layers as a variable-length list – which allows for some flexibility yet restricts the network topology search space – and genetic operators customised for specific layers.