Quasi-Recurrent Neural Networks with Embedded Variable Selection Module for Data Sequence Modeling in Dynamic Industrial Processes

Industrial processes usually exhibit nonlinearity and dynamic characteristics, whereby the data are subject to temporal variability due to continuous sampling. Thus, the prediction of quality variables in industrial processes relies on the utilization of dynamic models, such as quasi-recurrent neural network (QRNN). However, traditional QRNN cannot accurately extract the important feature containing predictive content from input variables during the modeling process. To complement this shortcoming, a novel data modeling method named quasi-recurrent neural network with embedded variable selection module (EVSM-QRNN) is proposed in this paper. First, a variable selection module (VSM) is embedded in a standard QRNN network, which replaces the common activation function to generate the alternative state of the cell. Then, the updated cell state incorporates additional valuable information of input variables, thereby attaining an equivalent effect as the conventional cell state update process. The proposed method effectively uses the features containing predictive information from input variables, thereby enabling accurate prediction of quality variables. Finally, the effectiveness of the proposed method is verified by simulation experiments on datasets collected from an actual industrial process.