TMEF-BI: Trusted Multimodal Evidential Fusion Considering Behavior Information for Fake News Detection

The multimodal fake news detection is challenging due to the wide scope of news topics and the frequent speed of emerging event updates on social media, which leads to the difficulty of discernible feature extraction of real and fake news. Furthermore, existing studies failed to consider the reliability of modalities and their fluctuation across different news. This poses an enormous challenge in attaining trusted multimodal fusion for fake news detection. The lack of uncertainty estimation, which shows the extent such decisions can be trusted, further compounds this issue. Thus, a novel method named trusted multimodal evidential fusion considering behavior information (TMEF-BI) was proposed, including the multimodal data evidence acquisition module, uncertainty estimation module, and trusted multimodal fusion module. It aims to: 1) acquire sufficient discernible features of real and fake news via speaker behavior information, which can provide domain invariant information beyond news content through their underlying state of mind and intentions; and 2) provide reliable decisions via uncertainty estimation. The experimental results suggest that the proposed method achieved an accuracy of 97.95%. Further experiments suggest that evidential fusion encompassed a deeper consideration of the uncertainty associated with each modality, leading to more reliable decision-making. This method sheds light on a fresh perspective on detecting multimodal fake news through reliability analysis.