Evaluation of ChatGPT and BERT-based Models for Turkish Hate Speech Detection

The popularity of large language models (LLMs) is increasing day by day. ChatGPT is one of the most popular LLMs. It is known for its success in many areas of natural language processing (NLP). Most importantly, we have yet to find zero-shot performance on various NLP tasks for low-level languages such as Turkish. Detection of hate speech is among the most important problems in NLP. With the growing social media usage, the prevalence of hate speech has also increased. However, automatic detection of hate speech in Turkish is rare compared to studies conducted in English. In our work, we analyzed the performance of ChatGPT and various fine-tuned BERT-based transformer models in detecting hate speech in Turkish. We found that ChatGPT provides similar results to the BERT-based models in detecting Turkish hate speech; thus, it is promising. In this study, a dataset consisting of 1000 Turkish tweets labeled “hate,” “aggressor,” and “none” was used.