Recurrent neural network training by nprKF joint estimation

We present a method for training recurrent networks with the joint estimation of states and parameters, using the "derivative-free" formulation for nonlinear Kalman filters by Norgaard, Poulsen, and Ravn (2000). Our approach is consistent with that described by Williams (1992) for the extended Kalman filter (EKF). We extend the treatment to handle multistream training and propose ways of making the required computation more efficient.