An Effective Approach for Neural Network Training Based on Comprehensive Learning

Multi-layer feed-forward neural networks have been used to tackle many complex practical applications. Their performance is closely related to the success of training algorithms which adapt the weights in the network. Although conventional algorithms such as back-propagation are widely used, they suffer from drawbacks such as a tendency to get trapped in local optima. Stochastic optimisation algorithms, and in particular population-based metaheuristics, represent a useful alternative in this context. In this paper, we have proposed an effective hybrid algorithm, CLPSO-LM, which is based on particle swarm optimisation (PSO), a population-based metaheuristic algorithm, the Levenberg- Marquardt (LM) algorithm as a local search algorithm, and a comprehensive learning (CL) strategy. The CL strategy in our algorithm is responsible for improving the exploration ability of the algorithm and preventing premature convergence using neighbour candidate solutions in PSO. The best position found by comprehensive learning PSO is then used as the initial network weights for the LM algorithm. An extensive set of experiments on different classification benchmark datasets and comparison to various conventional and population-based algorithms shows CLPSO-LM to yield very competitive performance.