Yours Truly: A Credibility Framework for Effortless LLM-Powered Fact Checking

In an era where social media portrays subjective realities, discerning truth from propaganda has become increasingly challenging. The proposed system addresses this issue with an end-to-end credibility framework to make fact-checking effortless and intuitive. Recognizing the subjective nature of claims, the system provides a robust method to assess the veracity of social media claims. Twitter, a key platform for public opinion exchange, influences cultural beliefs, political affiliations, and crisis responses. This work offers a pragmatic solution to navigate manipulative claims, reducing the cognitive effort to distinguish fact from fabrication and breaking misinformation chains sooner. Yours Truly uses FactStore, an extensive real-time database of fact-checked claims from Indian and International fact-checking initiatives. This database powers the search function, enabling time-sensitive searches with increased coverage to retrieve relevant context. The system breaks down compound sentences into atomic claims, verifying each with iterative context retrieval. Each claim is further fact-checked with multiple articles using text and semantic search. These matched articles are reranked for relevance using a technique called query-based committee selector. The top-ranked results provide context to an instruction fine-tuned Large Language Model, which infers the truth value of input claims. This approach tackles claims’ ambiguity and complexity and returns an interpretable credibility report explaining the inferred truth value. Yours Truly achieves an impressive F1 Score of 94% The framework is easily extensible to verify contents from other social media platforms, as it only relies on text without metadata and effectively handles long-form texts by atomizing compound statements. Yours Truly outperforms contemporary fact-checking systems on multiple misinformation baselines. It generalizes well across various text forms and information domains, demonstrating a high level of automation.