Sentiment Analysis Model Based on Self-Attention and Character-Level Embedding

Aiming at the problem of insufficient sentiment word extraction ability in existing text sentiment analysis methods and OOV (out-of-vocabulary) problem of pre-training word vectors, a neural network model combining multi-head self-attention and character-level embedding is proposed. An encoder-decoder (Seq2Seq) model is used for sentiment analysis of text. By adding character-level embedding to the word embedding layer, the OOV problem of pre-trained word vectors is solved. At the same time, the self-attention mechanism is used to perform attention calculations within the input sequence to find intra-sequence connections. The model uses bidirectional LSTM (Long Short-Term Memory) independently encodes the context semantic information of the input sequence to obtain deeper emotional features, to more effectively identify the emotional polarity of short text. The effectiveness of our method was verified on the public Twitter dataset and the movie review dataset published by ACL. Experimental results show that the accuracy of the model on the three categories of Twitter, movie reviews and IMDB datasets reaches 66.45%, 79.48% and 90.34%, respectively, which verifies that the model has excellent performance in datasets in different fields.