Deep Recurrent Stochastic Configuration Networks for Modelling Nonlinear Dynamic Systems

Deep learning techniques have shown promise in various domains. However, traditional methods can only demonstrate their universal approximation capability from an existential perspective, failing to explicitly construct such networks. This limitation is pronounced in nonlinear dynamic modelling, especially when faced with continual changes and uncertainties in the process, making them struggle to achieve accurate predictions and responses. This paper proposes a novel deep reservoir computing framework, termed deep recurrent stochastic configuration network (DeepRSCN) for problem solving. DeepRSCNs are incrementally constructed, with all reservoir nodes directly linked to the final output. The input random weight matrix and a special structure of the random reservoir weight matrix are assigned in the light of a supervisory mechanism, ensuring the universal approximation property. The output weights are updated online using the projection algorithm to handle the unknown dynamics. Given a set of training samples, DeepRSCNs can quickly generate learning representations, which consist of random basis functions with cascaded input and readout weights. Experimental results over a time series prediction, a nonlinear system identification problem, and an industrial data predictive analysis demonstrate that the proposed DeepRSCN outperforms other state-of-the-art methods with sound learning and generalization performance. Note to Practitioners—The motivation of this paper is to develop an advanced deep learning model with universal approximation capabilities for modelling nonlinear dynamic systems. While traditional methods can demonstrate infinite approximation of the target mapping from an existential perspective, they fail to explicitly construct the corresponding network. Furthermore, these approaches often encounter challenges, including unknown dynamic orders, arbitrary structure determination, and sensitivity to learning parameters, which can significantly impact the accuracy and reliability of industrial process analysis. To overcome these issues, the proposed DeepRSCNs are incrementally built using a supervisory mechanism, enhancing feature representation and maintaining universal approximation. Furthermore, DeepRSCNs can dynamically adjust the network parameters in response to real-time data, enabling them to adapt to changing operational conditions. These features are especially valuable in industrial environments, where capturing complex dynamics accurately and efficiently is essential for improving prediction performance and supporting better decision-making.