Email Spam Classification using DistilBERT

The increase of worldwide email users generated an estimate of 376.4 billion emails sent in a day. 85% of the sent emails were spam. To classify and filter increasing number of emails, Machine learning and Artificial Intelligence is applied. Distilled versions of transformer models for classification arise to perform on par with existing transformer models. It aims to be faster ang lighter. The objective of the study is to create a standalone device that identifies spam and ham using DistilBERT. The model is trained in several datasets with 11,214 ham and 7,287 spam emails. The device run in a flask server of a Raspberry Pi. It is inferenced with 100 spam and 100 ham emails with 95 spam and 97 ham emails correctly classified. The accuracy of the device is 95.99% with a MCC of 92.02%. It gained a positive correlation and prediction. Further areas of research are the improvement of training the model.