HateFusion: Harnessing Attention-Based Techniques for Enhanced Filtering and Detection of Implicit Hate Speech

Hate speech is a complex, multifaceted form of harmful material that targets individuals or groups. Detecting implicit hate speech poses a significant challenge due to its subtle expressions and the critical role of context in its identification. In response to this challenge, we have fine-tuned BERT and DistilBERT models on three publicly available datasets: Toxigen, Implicit Gab, and the Implicit Hate Corpus (IHC). Our approach involves meticulous data preprocessing, customized model configurations, and strategic hyperparameter optimization, all tailored to enhance the models’ performance in binary, 3-way, and 7-way implicit hate detection. We observed notable improvements in the detection capabilities of our proposed work. To enhance the explainability of model behavior, we visualized attention mechanisms, including attention heads and compatibility scores, using tools such as BertViz, local interpretable model-agnostic explanations (LIME), and Shapley additive explanations (SHAP). This facilitated the clear interpretation of model decisions. We developed a software application called HateFusion, which integrates our fine-tuned models with an intuitive graphical interface. The application includes a flagging option that allows users or moderators to mark highly hateful content, supporting continuous model refinement and updates. Our proposed model achieved the highest macro-weighted F1 scores across multiple datasets. On the IHC dataset (for 2-, 3-, and 7-way classification tasks), it achieved F1 scores of 0.75, 0.73, and 0.61 with DistilBERT, and 0.77, 0.72, and 0.60 with BERT. On the Toxigen (2-way) and ImpGab (3-way) datasets, it scored 0.77 and 0.89 with DistilBERT, and 0.82 and 0.89 with BERT, respectively.