Convergence of the generalized back-propagation algorithm with constant learning rates

A new generalized back-propagation algorithm which can effectively speed up the convergence rate and reduce the chance of being trapped in local minima was previously proposed by the authors (1996). In this paper, we will analyze the convergence of the generalized back-propagation algorithm. The weight sequences in generalized backpropagation algorithm can be approximated by a certain ordinary differential equation (ODE). When the learning rate tends to zero, the interpolated weight sequences of generalized back-propagation converge weakly to the solution of associated ODE.