Fcdnet: Fuzzy Cognition-Based Dynamic Fusion Network for Multimodal Sentiment Analysis

Multimodal sentiment analysis (MSA) provides a novel way to understand human sentiments. However, the differences between distribution patterns across modalities bring challenges in this domain. The inconsistency of recognitions with different modalities leads to incorrect final results. Moreover, the gaps between sentiments with different degrees are small in one modality, but the gaps between sentiments with same degree are large across different modalities. The imbalance leads to incorrect recognition for different sentiment degrees. Since the fuzzy network shows excellent performance in integrating data from multiple modalities, this study constructs a fuzzy cognition-based dynamic fusion network (Fcdnet) for MSA. The Fcdnet dynamically integrates sentiment scores across different modalities using a fuzzy cognition fusion mechanism, significantly enhancing the accuracy of identifying divergent sentiments across modalities. Additionally, a disparity balancing module is proposed to normalize the representations between different modality features by penalizing the similarity of sentiments with different degrees and rewarding the separability of sentiments with same degree. Experimental results demonstrate that Fcdnet outperforms state-of-the-art methods on public datasets, validating the superiority and effectiveness.