Fractional-Order RNNs: A Universal Approximation Framework for Non-Local Dynamic System Modeling

Recurrent neural networks (RNNs) endowed with continuous-time states have emerged as an adaptive framework for modeling dynamic systems. Among these systems, Caputo fractional-order ordinary differential equation systems (CFODEs) are gaining prominence due to their non-local characteristics over time. Building upon this, we propose an effective, novel, and interpretable state-space model, fractional-order RNNs (FORNNs), which exploits the non-locality of CFODEs. Firstly, this study theoretically demonstrates, for the first time, the capability of FORNNs to universally approximate CFODEs with arbitrary precision. Concurrently, during the application of FORNNs to practical scenarios, ‘Parameter Domain Problems’ (PDPs) and ‘Hallucination Problem’ are first revealed. In response to PDPs, we propose the Local Domain Normalization (LDN), along with introducing a novel loss function to rectify the Hallucination Problem encountered during the learning process. Finally, three real-world examples are presented to validate the superior performance of FORNNs, of which the results are entirely consistent with the theoretical proofs, and also confirm the effectiveness of the LDN method.