Fine-Tuned BERT Model for Hate Speech Detection in Political Discourse

This research tackles the challenge of detecting hate speech and offensive content in political discussions on social media. By employing natural language processing (NLP) techniques, the study aims to contribute to creating healthier online communities. The researchers started with a manually labeled dataset of Reddit comments centered on political discourse. To address the common issue of class imbalance in classification tasks, they used an oversampling strategy to ensure equal representation of each class. Additionally, they applied data augmentation, specifically synonym replacement, to increase the diversity of the training data, which helps improve the model's generalization. For text preprocessing, the Bidirectional Encoder Represen- tations from Transformers (BERT) tokenizer was used. This tokenizer converts text into subwords, which is especially useful for handling out-of-vocabulary terms. The model used in this study is a fine-tuned version of BERT base, trained with specific hyperparameters: a learning rate of 5e-5, a batch size of 24, and a weight decay of 0.01. Training was carried out over seven epochs using the Hugging Face Transformers library and NVIDIA Tesla P100 GPUs available on Kaggle. The model's performance was assessed using 5-fold cross- validation, resulting in impressive metrics: an average accuracy of 99.34 percent, precision of 99.36 percent, recall of 99.34 percent, F1-score of 99.34 percent, and an ROC-AUC score of 99.96 percent. These results highlight the model's effectiveness in identifying hate speech and offensive content across different data splits. Compared to earlier research, this approach shows significant improvement in performance metrics. It demonstrates the potential of combining BERT with data augmentation and oversampling techniques for detecting hate speech in political discourse on Reddit. This study lays the groundwork for future research into real-time detection systems and cross-lingual models.