Analyzing Sentiment Towards a Product using DistilBERT and LSTM

For improving customer experience towards a product, analyzing the customer’s review is the best possible solution, where customers’ positive and negative opinions can be identified. In our study, we have developed two different supervised natural language task models using (1) deep neural Long Short-Term Memory (LSTM) model and (2) advanced pretrained Distil Bidirectional Encoder Representations from Transformers (DistilBERT), where in the first model, we have analyzed how each of the words in a sentence contribute to the sentiment of that sentence, by training LSTM with our 50,000 Amazon reviews corpora, publicly available in the Amazon website. In the second model, we have examined how the positional embeddings of the words in a sentence and multi-headed self-attention help in faster with better sentence representation for sentiment analysis by fine-tuning DistilBERT, which contains 66 million parameters and 6 layers of encoders. The main contribution of our research includes- (1) a comparative analysis of the efficacy between these above mention models, where the Transformer based model outperforms the LSTM model in all metrics of evaluation, and (2) the establishment of a much faster and lighter sentiment analyzer model, DistilBERT.