Reddit Comment Toxicity Score Prediction through BERT via Transformer Based Architecture

Hateful and offensive language on social media platforms has a severe influence on users' mental health and engagement of people from various backgrounds. Automatic detection of foul language has traditionally relied heavily on datasets with categorical data. However, the degree of offensiveness of comments varies. The proposed model uses tfidf followed by Ridge Regression,Catboost Regression and BERT followed by dense layers. The study uses a dataset containing Reddit-comments written in English language with precise and calculated values ranging from -1 to 1. Best-Worst Scaling was used to annotate the dataset, a type of comparative annotation that has been found to reduce the biases associated with rating scales. It has been demonstrated that the technique gives extremely accurate offensiveness scores. The proposed method offers user to customize their own threshold of offensiveness. The experiments has been conducted with different n-gram ranges. The result reveals better performance than state of the art.