Overfitting by PSO trained feedforward neural networks

The purpose of this paper is to investigate the overfitting behavior of particle swarm optimization (PSO) trained neural networks. Neural networks trained with PSOs using the global best, local best and Von Neumann information sharing topologies are investigated. Experiments are conducted on five classification and five time series regression problems. It is shown that differences exist in the degree of overfitting between the different topologies. Additionally, non-convergence of the swarms is witnessed, which is hypothetically attributed to the use of a bounded activation function in the neural networks. The hypothesis is supported by experiments conducted using an unbounded activation function in the neural network hidden layer, which lead to convergent swarms. Additionally this also lead to drastically reduced overfitting by the neural networks.