Multimodal sentiment analysis based on TCN and cross-modal interactive feedback network

In response to the issues of noise and redundant information affecting the accuracy and robustness of existing multimodal sentiment analysis models, this paper suggests a multimodal sentiment analysis model based on TCN (Time-Causal Convolutional Network) and cross-modal interaction feedback networks. It is primarily composed of two modules: a modality fusion module and a feedback module. Firstly, the text, audio, and video feature vectors are encoded through TCN, followed by the generation of unimodal representations via dot-product attention. To capture interaction information across different modalities, a cross-modal fusion module comprising three symmetric multi-head attention mechanisms is introduced. To eliminate noise and redundant information generated from modality fusion, a feedback module consisting of TCN, fully connected layers, and sigmoid layers is designed to apply its generated masked features in the cross-modal fusion process. Finally, a fully connected layer is employed to integrate the concatenated vector of unimodal representations and cross-modal features, which is then fed into softmax for sentiment classification. Experimental results demonstrate that compared to existing models, this model exhibits superior performance in sentiment classification tasks.