Task-Aware BERT-based Sentiment Analysis from Multiple Essences of the Text

Text sentiment analysis has always been an important topic in the research of human-computer interactions and is generally applied to help businesses monitor product satisfaction and understand customer needs. The research in this study intends to consider the sentiment of the text with a focus on capturing multiple essences of the text, such as words, events and sentence, in a specific task. First, an approach to automatically extract the task-specific emotional key terms in the corpus of a specific task is proposed. Task-specific key events are manually/automatically designed for the specific application task. The BERT -based model is employed to integrate the outputs from sentence, key terms and events of the input text for task-aware sentiment analysis. The pre-trained sentence-based BERT model is fine-tuned using the Ren-CECps, a large-size Chinese weblog emotion corpus. Then we transfer the encoder weights to a new model and initialize a new linear layer, and finally fine-tune this model to fit the specific task. For evaluation, the Telecom Domain Customer Service Corpus (TD-CSC), a telecommunications service dataset, was used. The experimental results show that the proposed BERT -based model using multiple essences improved the correct rate by 10% compared to that without using the multiple essence features.