TriModal Enhanced Fusion Network: Advancing Multimodal Representation and Fusion for Enhanced Multimodal Intent Recognition

Multimodal intent recognition integrates language, expression, speech, and action to accurately determine intent. However, the completeness of feature information and the deep dependencies between different modalities remain challenges. This paper proposes the Tri-Modal Enhanced Fusion Network (TriMod-EFN) to enhance multimodal intent recognition. It is composed of Deformable Global-Local Transformer, Calibrated Fusion Module and Dual-Classifier. The Deformable Global-Local Transformer realizes the intra-modal information interaction for sequences of different lengths, and focuses on local detail information within the modality while acquiring the global information, thus acquiring richer semantic information. The Calibrated Fusion Module performs cross-scale and cross-modality semantic calibration, reducing semantic loss during modality dimensionality reduction and modality interaction, and learns dependencies with adaptive fusion weights. In the Dual-Classifier structure, the primary classifier and the auxiliary classifier are co-trained to reduce the prediction confusion. Experimental results on two datasets show that TriMod-EFN outperforms other advanced models in multiple evaluation metrics.