New block recursive MLP training algorithms using the Levenberg-Marquardt algorithm

A block formulation of the Levenberg-Marquardt algorithm to train feedforward MLPs is designed to track time-varying nonlinear functions. The resulting algorithm is called the block Levenberg-Marquardt algorithm. There are two varieties of the algorithm: the overlapping and the non-overlapping block Levenberg-Marquardt. The two algorithms are developed in terms of a block presentation of the input/output training set. The tracking problem can be viewed as one of solving a sequence of nonlinear identification problems. With the persistent excitation and slowly-varying system conditions satisfied, the Levenberg-Marquardt algorithm can be shown to have a uniform rate of convergence over the entire sequence of problems. The block Levenberg-Marquardt algorithms are tested on a nonlinear time-varying function tracking problem. The algorithms show performance that is superior to the performance of existing algorithms like the global extended Kalman filter algorithm with state noise in the system equations.