Robust Multimodal Sentiment Analysis Based on Adaptive Information Distillation and Adversarial Learning

Multimodal sentiment analysis (MSA) has exhibited superior performance compared to unimodal sentiment analysis. However, in real-world scenarios, multimodal data are prone to being incomplete due to various interferences. To address this issue, this paper combines cross-modal adaptive knowledge information with adversarial reconstruction of complete-incomplete multimodal representations, proposing a novel neural network model called Dynamic Information Transfer-Adversarial Learning (DIT-AL) for robust MSA of unaligned modal sequences with missing data. First, the model employs a symmetric structurebased attention feature fusion to facilitate local and global feature interactions across modalities, learning multimodal representations for complete-incomplete data pairs. Then, two mechanisms are utilized simultaneously to recover the semantic information of the missing data. The first mechanism dynamically learns the relative relationships between complete and incomplete modal representations through an adaptive information distillation module, bridging the semantic gaps caused by missing modal data and enhancing the representation capacity of incomplete multimodal representations. The second mechanism introduces adversarial learning and feature reconstruction based on paired multimodal representations to generate semantic information for incomplete data, thereby achieving consistency in the feature distribution between complete and incomplete data. Experimental results on three public benchmarks (CMU-MOSI, CMU-MOSEI, and CHSIMS) demonstrate that the proposed method can effectively improve the robustness of MSA under random data missing conditions by using adaptive information distillation combined with adversarial reconstruction. Compared with state-of-the-art methods, it achieves higher recognition accuracy for robust MSA