Evaluation of Chinese Sentiment Analysis APIs Based on Online Reviews

Over the last decade, online reviews on social media platforms have become one of the most influential information sources. With the help of sentiment analysis tools, this unstructured information can be converted into structured data and analyzed to extract consumers’ opinions on products and services to assist important business decision-makings. Due to its great practical value, sentiment analysis using technologies like artificial intelligence has attracted attentions of researchers and practitioners. Many major Internet companies have developed their application program interface (API) to provide sentiment analysis services for individuals and businesses. This study aims to test and evaluate current mainstream Chinese sentiment analysis applications. First, authentic online reviews are collected from Ctrip.com, a major online travel Agency in China, using a crawler. These reviews are then analyzed using major Chinese sentiment analysis APIs. Meanwhile, some interviewees were asked to classify and rate these reviews as positive, neutral or negative as well using a survey. After that, human-API discrepancy is compared and evaluated. Our results revealed that classification errors of these APIs are mainly caused by incorrect word segmentation and fail to integrate the context into semantic interpretation.