Fake News Detection With Credibility Scoring on Turkish Twitter Data

The widespread use of social media tools today has led to a rapid increase in newsworthy data on the internet. Some of the news data may contain inaccurate content, and the repeated sharing of these by users can lead to significant problems. Preventing these problems before they happen is very important today. Automatically detecting data referred to as fake news can help prevent such situations. In this study, fake news detection was performed using deep learning methods on tweets and their credibility scores from a Turkish Twitter news dataset. As a deep learning model, a fine-tuned version of the large language model llama-2-7b, called “mohammedbriman/llama-2-7b-chat-turkish-instructions,” was used for Turkish instructions. Four models were fine-tuned for four different scenarios, and results were obtained. Additionally, to achieve higher performance, considering the resource needs and response times of large language models, training was also conducted with the smaller language model Phi-3 (Phi-3-mini-4k-instruct) in four scenarios, and the results were analyzed. In the developed models, F1 scores ranged from 0.74 to 0.99. In another study that also used deep learning methods for fake news detection in Turkish, the models achieved F1 scores between 0.828 and 0.947. When comparing the performance of the models we designed with this study, it was concluded that our models performed better. Furthermore, these results show that the developed models are effective in detecting fake news on social media platforms and represent an important step in protecting users from misleading content.