A Dual Averaging Algorithm for Online Modeling of Infinite Memory Nonlinear Systems

An online modeling algorithm is derived from a generic stochastic dual averaging (DA) method. It employs a negative entropy as a distance-generating function and the Volterra series expansion as a dictionary. Assuming that the measurement data are not i.i.d. but generated by a nonlinear dynamical system with an infinite, exponentially fading memory, the error bounds are established for both the generic DA method and for the proposed modeling algorithm. The experiments performed on a set of benchmark systems confirm the applicability of the algorithm in real-world scenarios and demonstrate its low computational complexity.