Prediction of key chemical parameters based on improved Transformer

It is difficult to build accurate prediction models for complex chemical processes with significant nonlinearity and dynamics based on traditional shallow static models. A prediction model based on improved Transformer called CNN-Trans is proposed to address the above challenges. In order to improve the efficiency of Transformer and enhance its local feature extraction capability, the CNN architecture is applied to Transformer's architecture: (1) Dilated causal convolution is used in the embedding layer to obtain multi-scale information and a larger sensory field. (2) Replacing Transformer's feed-forward neural network (FFN) with productive mixed attention (PMA), combining temporal attention and channel attention, significantly reducing model complexity while maintaining performance. (3) The convolution-pooling module is added to the stacking layers of the Encoder to fuse the obtained multi-scale features to obtain finer-grained information. The proposed method was applied to the actual data of PVC drying in a chemical company, and satisfactory results were obtained for the prediction of water content, indicating the feasibility of the proposed method in practical chemical applications.