Improved Fake News Detection by Combining Sentence Transformers, Variational Autoencoders, and Topic Modelling: the VAE-Topic Model Fusion Method

In this study, we present a pioneering method aimed at discerning the veracity of news content by leveraging state- of-the-art embedding techniques. Recognizing the importance of capturing semantic nuances, we employ sentence transformers to generate embeddings that encapsulate the essence of the news content. This process results in high-dimensional data, which, although rich in information, can be computationally intensive for subsequent analysis. To address this, we introduce a Variational Autoencoder (VAE) to achieve an effective dimensionality reduction while preserving critical information. However, given the intricate and hierarchical nature of news content, we integrate a topic modeling approach known as Pachinko Allocation Model (PAM) into our framework. PAM offers a more refined data representation by capturing the latent hierarchical relationships between various news topics, adding an extra layer of semantic understanding to our method. The enhanced data representations from the VAE and PAM processes are con- catenated, creating a comprehensive feature set. This combined feature set is then used to train multiple classifiers to segregate real news from fake counterparts. Through this comprehensive numerical investigation, we have achieved an accuracy of 96.80% and an F1 score of 97% using the transformer-based sentence paraphrase- multilingual architecture. Our study’s findings are not only significant in terms of accuracy but also underscore the potential of combining multiple advanced techniques to tackle the fake news challenge.