Sentiment Analysis using DistilBERT

Transformers is an architecture that performs well in NLP task. To understand and improve its performance on sentiment analysis, DistilBERT is employed as the base model. Sentiment analysis is a process that extracts subjective information from textual data and categorizes them into different classes. The classification classes may include polarity (positive, neutral, negative) or emotions (happy, sad, angry). In addition, multiple techniques such as fine tuning, regularization and hyperparameter tuning are applied to improve the performance of the model. The proposed solution acquired an accuracy score of 85.41% on Internet Movie Database (IMDB) dataset and 86.59% on Customer Reviews (CR) dataset.

I. Introduction

Transformers is a deep learning architecture that has achieved good results on deep learning task. [1] To understand and improve its performance on sentiment analysis, DistilBERT is employed as the base model architecture to perform sentiment analysis. Sentiment analysis is a common NLP task which extracts information from textual data. While this subtask might not be as useful as it seems at first glance, it is actually applied by many huge companies to gather information from users.