Siamese Neural Network and Multimodal Data Fusion Approach for Small-Sample Learning in Industrial Soft Sensor Modeling

In industrial scenarios, soft sensor modeling often faces the challenge of underfitting due to limited available data. Therefore, in the field of industrial soft sensing, it is crucial to develop solutions for regression problems based on small-sample learning. One potentially feasible approach is to pair the data used for modeling to generate a larger set of data pairs, and then utilize Siamese neural networks (SNNs) to extract additional information from the paired data, thereby addressing the underfitting issues. Moreover, the integration of data from multiple sources can also help alleviate the problem of underfitting. Based on these ideas, this article proposes an SNN-based multimodal data fusion framework (SNN-MMDFF) for tackling the challenges of small-sample learning in industrial soft sensor modeling. The SNN-MDDFF employs end-to-end multimodal feature extraction (MMFE) networks to extract multimodal features from the paired multimodal data sequences, which are then employed to predict the soft sensing target. A three-stage training strategy, which incorporates hierarchical metric learning and component-by-component training approaches, is implemented to train this framework. The SNN-MMDFF is validated in a case study on cement clinker free calcium oxide (f-CaO) content soft sensing, which demonstrates superior performance and robust modeling stability.