Multimodal Sentiment Analysis Model Based on Text-Guided and Cross-Modal Feature Fusion

Multimodal Sentiment Analysis (MSA) is a method that synthesizes information from different modalities to accurately identify expressed emotions. In MSA, textual modality usually carries more direct sentiment information, thus it occupies a key position in sentiment prediction. In this paper, we propose a text-guided and cross-modal feature fusion multimodal sentiment analysis model HTFF based on how to effectively fuse nonverbal and textual modalities while solving the problems of information redundancy and ambiguity of sentiment expression in nonverbal modalities. The model processes the modal features through the feature extraction embedding module and utilizes the collaborative attentiveness hyper-modality generator module to extract the visual, audio, and audio-visual under the guidance of language features. hyper-modality representations for mixed modalities. A complementary joint representation is formed by a Transformer-based cross-modal feature fusion module, which ultimately realizes emotion prediction. HTFF was evaluated on the CMU-MOSI and CH-SIMS datasets, results show that compared with other advanced models, HTFF has better overall performance.