Multimodal Emotion Analysis Based on Multi-Scale Feature Fusion and Cross-Modal Attention: Architecture Comparison and Feature Extractor Optimization

Multimodal emotion analysis can capture and understand human emotions more comprehensively by integrating information from different modalities, thereby improving the accuracy and robustness of emotion recognition. However, existing approaches often suffer from feature redundancy and low modal interaction efficiency in multimodal sentiment analysis. To address these challenges, this paper proposes a deep learning framework that integrates multi-scale features and cross-modal attention mechanisms. We conduct a systematic evaluation of five fusion architectures (CMAC, OTE, HSTEC, NaiveCat, NaiveCombine) and multiple feature extractors (InceptionV3, WideResNet50, MobileNetV2, ResNet50/101, BERT/RoBERTa). Extensive experiments demonstrate that: 1) The combination of BERT and WideResNet50 achieves the optimal balance between accuracy and computational efficiency; 2) The CMAC framework significantly outperforms baseline models through hierarchical feature fusion, with its adaptive weight allocation mechanism effectively mitigating overfitting; 3) Channel attention mechanism exhibits the best performance in text feature extraction. Moreover, ablation experiments further confirm that text features are the dominant source of information.