Sentiment Analysis of Game Reviews on STEAM using BERT, BiLSTM, and CRF

The Steam platform releases thousands of games almost every week, providing customers with many options, which require various considerations to make a purchase decision. Steam has features that can help consumers consider purchasing games, such as the help dashboard that provides information on game reviews and ratings. However, based on user input reviews, game ratings are still manual, which may cause human error and inaccurate results. To help gamers get the desired game information and increase customer purchasing power towards available games, sentiment analysis is one technique for classifying opinions. This study focuses on sentiment analysis of game reviews, which will result in classification into two classes, positive and negative. It will be measured for its performance based on the f1 score evaluation results. The proposed model that we created is to compare sentiment analysis models using the baseline Bi-directional Long-short Term Memory (BiLSTM), Bidirectional Gated Recurrent Unit (BiGRU), and fine-tuning Bidirectional Encoder Representations from Transformer (BERT). We then propose another model, BiLSTM-CRF, by examining the effect of using the last four hidden states of BERT and comparing it with the last one hidden state of BERT to be used in the BiLSTM-CRF model. Based on the results of sentiment analysis models we developed for the ten most reviewed games on Steam, the BiLSTM model combined with CRF on the last four hidden states yielded the best performance, with an Fl score of 95.2 %. This score exceeds the baseline transformer-based model's Fl score of 88.0%. This is because combining BiLSTM and CRF allows the log-likelihood of the BiLSTM emission sample to be obtained more accurately so the model does not lose its context.

I. Introduction

Steam is a successful game distribution platform with the largest video game sales transaction online. According to Linaer, Steam currently has 1 billion accounts, with 90 million accounts considered active users [1]. The Steam platform releases thousands of games almost every week, causing various considerations on the customer's side to decide on purchasing a game [2]. One way to consider purchasing a game is by reading game reviews, but with the number of reviews reaching thousands, the ability to interpret opinions is essential for consumers.