A Multimodal Fusion Framework for Fake News Detection via Multi-Attention Mechanism

Social media platforms have emerged as the primary channels for the general public to access and share information. However, the rapid dissemination of news has led to the emergence of a significant amount of unverified content, posing a serious threat to media credibility and network security. The current solutions primarily focus on detecting fake news by extracting and fusing features from images and text, but they have not fully utilized the relevance within and between modalities, resulting in suboptimal fusion effects. In this paper, we propose a multimodal fusion framework via multi-attention mechanism (MFMA), which considers not only the semantic and relevance features of images, textual content, and Optical Character Recognition (OCR) text from the news, but also the multimodal relevance features between them. First, we extract text within images (OCR text) to address the issue of inadequate utilization of the visual modality in traditional methods. Second, we select more robust feature extractors to capture the independent characteristics of each modality and use a pre-trained FcaNet model to extract more comprehensive image quality features. Additionally, two relevance extraction modules are designed to achieve feature fusion within and between modalities. Finally, the combined features are fed into a classifier to determine the authenticity of the news. The experimental results and analysis indicate that the model we proposed effectively enhances the performance of fake news detection.