AL-HCL: Active Learning and Hierarchical Contrastive Learning for Multimodal Sentiment Analysis with Fusion Guidance

Multimodal sentiment analysis (MSA) is a rapidly advancing field in artificial intelligence (AI). However, it faces two major challenges: (1) deep learning-based MSA models often rely on large multimodal datasets but struggle with suboptimal data utilization, and (2) inconsistencies across modalities hinder the effective fusion of diverse information sources. To address these challenges, we propose the Active Learning and Hierarchical Contrastive Learning (AL-HCL) model for MSA. This model incorporates active learning techniques to balance prediction uncertainty with sample diversity, selectively identifying and labeling high-value samples from an unlabeled pool. This approach reduces annotation costs while maintaining robust performance. Additionally, we introduce a three-tier contrastive learning framework. The first layer addresses heterogeneity within unimodal data, the second resolves discrepancies between unimodal and fused modalities, and the third employs a Matrix-Based Fusion (MBF) module to extract high-level semantic features, enabling deeper feature-level fusion. A novel modal fusion strategy further enhances cross-modal interactions, optimizing the fusion process. Extensive experiments on benchmark MSA datasets—CMU-MOSI, CMU-MOSEI, and CH-SIMS—demonstrate that AL-HCL outperforms state-of-the-art models, validating the effectiveness of the proposed active learning strategy.