Complete memory structures for approximating nonlinear discrete-time mappings

This paper introduces a general structure that is capable of approximating input-output maps of nonlinear discrete-time systems. The structure is comprised of two stages, a dynamical stage followed by a memoryless nonlinear stage. A theorem is presented which gives a simple necessary and sufficient condition for a large set of structures of this form to be capable of modeling a wide class of nonlinear discrete time systems. In particular, we introduce the concept of a "complete memory". A structure with a complete memory dynamical stage and a sufficiently powerful memoryless stage is shown to be capable of approximating arbitrarily wide class of continuous, causal, time invariant, approximately-finite-memory mappings between discrete-time signal spaces. Furthermore, we show that any bounded-input bounded output, time-invariant, causal memory structure has such an approximation capability if and only if it is a complete memory. Several examples of linear and nonlinear complete memories are presented. The proposed complete memory structure provides a template for designing a wide variety of artificial neural networks for nonlinear spatiotemporal processing.