Convergence Study in Extended Kalman Filter-Based Training of Recurrent Neural Networks

Recurrent neural network (RNN) has emerged as a promising tool in modeling nonlinear dynamical systems, but the training convergence is still of concern. This paper aims to develop an effective extended Kalman filter-based RNN training approach with a controllable training convergence. The training convergence problem during extended Kalman filter-based RNN training has been proposed and studied by adapting two artificial training noise parameters: the covariance of measurement noise (
R
) and the covariance of process noise (
Q
) of Kalman filter. The
R
and
Q
adaption laws have been developed using the Lyapunov method and the maximum likelihood method, respectively. The effectiveness of the proposed adaption laws has been tested using a nonlinear dynamical benchmark system and further applied in cutting tool wear modeling. The results show that the
R
adaption law can effectively avoid the divergence problem and ensure the training convergence, whereas the
Q
adaption law helps improve the training convergence speed.