Adaptive Attention-Driven Manifold Regularization for Deep Learning Networks: Industrial Predictive Modeling Applications and Beyond

Industrial predictive modeling, which provides valuable information for process monitoring and decision-making on process operation, plays a crucial role in the process industry. However, industrial processes commonly exhibit nonstationary characteristics caused by various process drifts, such as frequent variations in the properties of raw materials. Hence, this article proposes an adaptive attention-driven manifold regularization (AAMR) strategy. Specifically, it designs an adaptive working condition selection strategy to overcome sudden variations in the raw material properties. In addition, a novel attention distance calculation is introduced to minimize the impact of noise and redundant features, which aims to address the limitations of conventional manifold learning distance calculations. Finally, the proposed manifold regularization strategy is fused into the stacked autoencoder (SAE), coined AAMR-SAE, to enhance dynamic manifold feature extraction capability and strengthen the parameter update process. Two real industrial applications are presented to verify the efficacy of the proposed method. The results confirm that the proposed method can provide superior prediction accuracy and practicality compared to some existing representative methods.