A successive overrelaxation backpropagation algorithm for neural-network training

A variation of the classical backpropagation algorithm for neural network training is proposed, and convergence is established using the perturbation results of Mangasarian and Solodov (1994). The algorithm is similar to the successive overrelaxation (SOR) algorithm for systems of linear equations and linear complementary problems in using the most recently computed values of the weights to update the values on the remaining arcs.