Multimodal Fake News Detection Based on Contrastive Learning and Cross-Attention Mechanism

With the development of the internet, a vast amount of maliciously fabricated fake news is rampantly spread across various social media platforms, severely disrupting the public's perception of facts and even having a significant adverse impact on the authenticity of internet information. Natural Language Processing (NLP) models have been widely applied in the field of fake news detection, and the use of NLP technology to detect fake news has become a hot research topic both domestically and internationally. With the advancement of image editing and image generation technologies, fake images are increasingly flooding internet news, making the detection of fake news based on solely textual modality more challenging. This has made multimodal fake news detection increasingly important. Existing multimodal fake news detection faces two issues: the inability to effectively fuse features from image and text modalities, and the inability to enable models to learn the consistency and differences within and between modalities to detect false information. To address these issues, cross-attention mechanisms and contrastive learning have been introduced into multimodal fake news detection, enhancing the fusion of multimodal features and the effectiveness of fake news detection. Firstly, the cross-attention mechanism is used to deeply model information from different modalities, achieving efficient fusion of multimodal information. In addition, two new methods for constructing positive and negative examples in multimodal fake news detection were proposed, using contrastive learning to semantically align images and text. Experiments show that the proposed model improved by 5.9%, 4.4%, and 1.6% respectively on three real-world datasets compared to the state-of-the-art (SOTA) models, and the effectiveness of each proposed module was verified through ablation studies.