Mafe: Multi-modal Alignment via Mutual Information Maximum Perspective in Multi-modal Fake News Detection

With the rapid advancement of social media, the spread of fake news in multi-modal forms has become increasingly prevalent, causing not only individual misguidance but also disruption to the social order. Many researchers have proposed methods that utilize both visual and textual information for fake news detection. Although researchers have made such valuable attempts, current methods still struggle due to the following crucial challenges: calculating the relevance of different modalities and comprehending their influence on decision-making in the context of fake news detection. To overcome these challenges, we propose Mafe —a Multi-modal Alignment FakE news detection method via mutual information maximum perspective. By jointly modeling the multi-modal context information on a multi-modal alignment module, we present mutual information maximum perspective to align different modalities. To achieve this, we leverage text encoder and image encoder to learn better representations for text and images, respectively. Subsequently, these features are fed into the multi-modal alignment module to capture interactions between the two modalities and compute their relevance. Finally, an attention mechanism is introduced to weigh the features, guiding the stages of feature-fusing and decision-making. The effectiveness of the Mafe model is demonstrated through extensive testing on two real datasets.