Chinese Sentiment Analysis Based on Lightweight Character-Level BERT

With the emergence of more and more deep learning application scenarios, training a complete model from scratch in various scenarios is a tedious and costly task, so transfer learning has received more and more attention. We all know that sentiment analysis as an effective means of acquiring user behavior has been widely used in various social or e-commerce platforms in the field of natural language processing. However, the traditional Chinese sentiment classification model is constructed based on word vectors due to the limitation of the language characteristics of Chinese itself, so it is necessary to do some preprocessing of word segmentation in advance. The performance of the word segmentation directly affects the performance of the model and training a complete model from scratch requires sufficient computing resources. In this paper, we propose a character level transfer learning model built on a lightweight pre-trained BERT model, here we named it CTBERT. On the basis of fully absorbing the attention mechanism of Transformer, we fine-tuned and trained CTBERT, and then compare CTBERT with traditional machine learning and deep learning models. The results show that our transfer model has a significant performance improvement over existing model.