Sentiment Analysis of Danmaku Videos Based on Naïve Bayes and Sentiment Dictionary

Danmaku video provides a platform for users to communicate online while watching videos. Danmaku is a live commenting function where the comments related to the video being screened are created by users and prominently shown in real-time on the video screen. These live comments contain complex and rich sentiments, reflecting users’ instant opinions and feelings on video programs. In some sense, danmaku provides emotional timing information about video data, and it also offers an innovative mean to analyze video data. However, existing sentiment classification methods are not suitable for danmaku data analysis. To solve this problem, this paper constructs a danmaku sentiment dictionary and presents a new method using sentiment dictionary and Naïve Bayes for the sentiment analysis of danmaku reviews. The method is greatly helpful in supervising the overall emotional orientation of a danmaku video and predicting its popularity. Through the processes of extracting emotional information from a danmaku video, classifying sentiment and visualizing data, the time distribution of the seven sentiment dimensions can be obtained. In addition, a weight calculation can be conducted for classifying the sentiment polarity of danmaku reviews. Experimental results show that the proposed method has a significant effect on sentiment score and polarity detection.


I. INTRODUCTION
With the growing popularity of online social networks and the booming development of the internet, danmaku, a new media, has recently emerged. ''Danmaku'' originated from Japanese Niconico animation video website, which was introduced by Chinese video websites AcFun and Bilibili in 2008. Danmaku is a live commenting function in which the comments about a video are synchronized with the video content. Each comment is associated with the specific time and content in the video. The basic format of danmaku is illustrated in Fig. 1. As we can see, danmaku is displayed directly on the video interface in the form of sliding text. Danmaku flows on the top portion of the screen and can be represented in different colors and fonts. In danmaku video mode, users choose different headshots to represent themselves, and they can see and reply to the previous users' danmaku while watching the video. Currently, with the rise of danmaku, an increasing number of The associate editor coordinating the review of this manuscript and approving it for publication was Shouguang Wang . Chinese mainstream video websites, such as iQIYI, Tencent Video, and Mango TV have introduced a danmaku mode.
As a new way of sharing messages, danmaku writing is very different from traditional short texts in two aspects. First, the text of danmaku is shorter and more casual, and most VOLUME 8, 2020 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ commenters choose phrases and symbols to express their thoughts and feelings. Second, the content is more colloquial and symbolic, and users often express emotions using network catchwords. Essentially, danmaku text is short text based on a time series in a video, and each danmaku corresponds to specific point in time in the relevant video.
Although there is a great deal of research on short texts, most studies are based on social media content and focus on individual senders, experiences and content sharing. In the danmaku system, however, the short text is centered around a specific point in a given video. Users post comments related to the video's content, including their impressions and evaluations of the content, as well as comments full of ridicule and more. This live data is more beneficial for analyzing video content and user feedback in real-time.
In some senses, danmaku video relieves the sense of distance in space and time among users. It effectively meets users' social communication needs in the process of watching videos. From there, users can express emotions and interact with others online and thus foster the feeling of companionship. However, due to the differences in users' viewing psychologies, education levels, and theme preferences, there is uncertainty in online video demands. This uncertainty is often manifested in danmaku, as it reveals the differences between the expectations for and actual experiences with online videos. In addition, danmaku video leaves digital footprints of communication, which can be used by other users as references for video content. Consequently, it is significant to study the sentiment analysis of danmaku reviews, using big data to analyze users' emotional tendencies, discover their reactions and attitudes to different video content [1], and then obtain the satisfaction evaluation [2]. This sentiment and evaluation information can provide a reference for other users when choosing a video program. In this way, the opinions and emotional expressions in danmaku reviews can be more popular and referential.
Considering the above background and reasons, how to utilize the characteristics of danmaku to design a sentiment analysis model, and how to investigate the dynamic changes in multi-dimensional sentiment in danmaku are new challenges in sentiment evaluation. In this paper, based on mining the danmaku data in semantic layers and statistics analyzing users' sentiment features, a sentiment analysis model of danmaku reviews is constituted. The study proposes a hybrid method using sentiment dictionary and Naïve Bayes to extract sentiment features to analyze danmaku video content. This hybrid analysis approach is a hot research topic in sentiment analysis that can improve the generalization performance of sentiment classification. Hence, the results of this study are of significance and have practical application value for set a new milestone in the field of video content personalization.
In particular, the contributions of this paper include the following: • We construct a danmaku sentiment dictionary that includes the Chinese emotional vocabulary ontology database of the Dalian University of Technology (DUT), a catchwords dictionary and a set of emoticons. The emoticon set is built using a large danmaku dataset.
• We propose an effective hybrid feature extraction method that utilizes the sentiment dictionary and Naïve Bayes to identify and classify the multi-dimensional sentiments of danmaku messages.
• We propose an overall framework for sentiment value calculation. Considering the characteristics of Chinese sentence expression, the sentiment words are modified by negative words and degree adverbs. We summarize these modifiers and conduct a weight calculation for estimating the sentiment polarity of danmaku reviews.
• We implement a sentiment analysis of real-world danmaku datasets, obtain the dynamic changing trend of the number of danmaku messages and their timestamps, and obtain the time distribution of multi-dimensional sentiments of danmaku messages.

II. RELATED WORK A. RESEARCH ON DANMAKU REVIEWS
With the recent popularity and wide application of natural language processing technology, studies on short text, which have been going on for years and some of which are based on social media platforms such as Twitter and Facebook, have also matured [3]- [7]. As a new way of commenting, danmaku is a text-based video-commenting function based on short text, closely related to video content and synchronized with time stamps. At present, relevant studies on danmaku reviews mainly focus on video content analysis, which is manifested in three aspects: highlighted video shot extraction, popularity prediction, and automatic marking. For instance, Lv et al. [8] proposed a time-depth structured semantic model (T-DSSM) in which a video is segmented by clustering, and the theme distribution of video clips can then be obtained. Through supervised learning and identifying semantic vectors, the extraction of video highlights is achieved. He and Ge [9] proposed a video popularity prediction model in which danmaku is measured from multiple aspects. Since the uploader and video quality can affect the popularity of danmaku videos, the study combines these two factors and herd effect to predict the popularity of danmaku videos.
''Highlight'' video clips are detected by users' discussion of certain video content with strong sentimental color [10]. Zhuang and Liu [11] proposed the AT-LSTM model to identify emotional keywords in danmaku reviews. The model can more effectively extract the theme-based "highlight" video clips, which helps users accurately retrieve useful online information. In the work of Wu et al. [12], both user interest and user relation are taken into consideration in a personalized theme model. By extracting keywords from the theme model to mark video shots, the noise caused by short danmaku review text is solved. Deng et al. [1] constructed a classification of danmaku words based on the Latent Dirichlet allocation topic model and evaluated the multi-dimensional emotion vectors of danmaku. According to the emotional relationships among a set of video clips, the recommendations of emotional clips of each video was analyzed.

B. APPROACHES TO SENTIMENT ANALYSIS
Sentiment analysis is a process of judging emotion expression through language, which includes the extracting emotional information, processing and analyzing data, and classifying the sentiment of selected text [13]- [15]. The basic tasks of sentiment analysis are emotion recognition, polarity detection and affective computing [16]- [18]. Text sentiment analysis is a key technique in the field of natural language processing for emotion-mining, which is widely used in public opinion monitoring, AI and business intelligence [19]. There are three main types of approaches for text sentiment analysis: a machine learning-based approach, a dictionary-based approach and a hybrid approach [20]- [21].
In machine learning-based approaches, a sentiment classifier is trained using a pre-labeled corpus. A classifier can be built to determine the polarity of a text document with algorithms, such as Naïve Bayes (NB), Support Vector Machine (SVM), Maximum Entropy (Max Ent) and Word2vec, etc, which are commonly used in sentiment analysis [22]- [25]. For instance, Ye et al. [23] compared the sentiment classification effects of NB, SVM and N-gram model on text comments and found out that the accuracy of SVM and NB algorithm is significantly higher than that of N-gram model. Considering the semantic relationships between words, Zhang et al. [24] proposed a method based on Word2vec and SVMperf to classify Chinese comment texts. The experimental results demonstrate the effectiveness of this hybrid algorithm, which can reach over 90% accuracy in sentiment classification. In the work of Yang et al. [25], a segment-level joint topic-sentiment model (STSM) is designed to capture topic-sentiment correlation and estimate fine-grained sentiments. To get the best sentiment classification results, various classification models have been developed in recent years: a semi-supervised sentiment model [26]- [27] and deep learning models based on neural networks [26]- [32], including a hybrid ensemble pruning model [28]- [29], bag of meta-words model [30]- [31], and a long short-term memory (LSTM) model [32]- [34].
Within the dictionary-based approach, a pre-developed dictionary containing the polarity of words or phrases is used to calculate the sentiment scores and determine the polarity of a text document [35]. Sentiment calculation mainly relies on open source or extended sentiment dictionaries [36], and the sentiment value of a sentence can be computed according to various semantic rules [37]. The existing English sentiment dictionary includes Word-Net-Affect [38] and Senti-WordNet [39]. The Chinese sentiment dictionary includes HowNet [40], the Chinese emotional vocabulary ontology database of the Dalian University of Technology [41], and a Catchwords dictionary [42]. In the studies of Denecke [43] and Ohana and Tierney [44], SentiWordNet was used to determine the polarity and sentiment classification of text documents. Thompson et al. [14] adopted the semantic orientation calculator in sentiment analysis of player chat messaging in video games. The results show that dictionary-based sentiment extraction is an efficient method in sentiment classification. Based on opinion target extraction, Wu et al. [45] proposed a method to automatically construct a target-specific sentiment dictionary. An unsupervised algorithm was used to extract opinion pairs, and a framework was established to classify their sentiment polarities.
The third approach is a hybrid approach that combines the first two approaches together for sentiment classification. Generally, the machine learning-based approach is more accurate, but it needs much more time to label the data [46]. In contrast, the dictionary-based approach has advantage of requiring no training data to classify the sentiment, and its computing time is much faster than that of machine learning [47]. To improve the accuracy and efficiency of sentiment classification system, the dictionary-based method is combined with the machine learning-based approach [48]. Bandhakavi et al. [49] extended domain specific emotion lexicon (DSEL) generation, based on which, the authors developed an emotion feature extraction. Their study shows that the unigram mixture model can be used to effectively extract features for sentiment classification. Zhu et al. [50] developed an improved ensemble learning method for sentiment classification that integrated Word2Vec, SVM and a sentiment dictionary. The experimental results show that the ensemble learning method can efficiently enhance the precision of sentiment classification.
For the Chinese sentiment analysis research, Peng et al. [51] discussed sentiment classification methods for the Chinese language, such as machine learning-based approaches, knowledge-based approaches and mixed models, and tested these approaches with various Chinese datasets. Huang et al. [52] proposed a new ensemble learning framework for the sentiment classification of Chinese online reviews. By using the random subspace classifier based on product attributes, the sentiment information of online reviews could be obtained. Chen and Huang [53] utilized knowledge-enhanced neural networks for the sentiment analysis of Chinese reviews. In the study, they made use of aspect-level sentiment classification to identify aspect-opinion pairs and determine sentiment polarities. Li and Liu [54] presented an unsupervised method CHOpin-ionMiner that extracts sentiments or opinions about a subject from online Chinese reviews. A method based on phrase structure grammar was proposed to extract the largest noun phrase as the candidate opinion target.
On the basis of the related work above, this paper uses a hybrid approach to danmaku sentiment analysis. That is, we use a sentiment dictionary to extract the features from danmaku text, and then adopt a Naïve Bayesian algorithm for sentiment classification. The hybrid approach saved computing time and decreased the labeling cost. It also raised the accuracy of the sentiment analysis by applying a machine VOLUME 8, 2020 learning technique. In this study, the sentiment analysis of danmaku reviews is mainly based on a danmaku dictionary, which is constructed by an expansion of the basic sentiment dictionary. Then, the sentiment polarity of danmaku reviews is calculated according to the weight of the sentiment words. Finally, the dynamic changing trend of multidimensional sentiment of danmaku with time is obtained.

III. SENTIMENT ANAYLSIS OF DANMAKU
Danmaku has changed the traditional one-way output video play mode, enabling real-time communication between viewers and video content creators. Danmaku makes viewing videos a lively and interesting activity through which viewers may convey information and share emotions with others. Therefore, users' viewing experiences have been greatly improved. Compared with traditional comment data, danmaku data contains comment text and corresponding video timestamp information, and it can thus more accurately and specifically reflect users' immediate emotions, as well as their positive and negative comments. Danmaku sentiment information is helpful for users to satisfy their information needs, such as specific video types and content, when selecting videos. Based on the research in the abovementioned works, we elaborated a dictionary with emoticons and numerical strings as items and added them in the text data to identify the multi-dimension sentiment of danmaku reviews.

A. CONSTRUCT A SENTIEMNT DICTIONARY FOR DANMAKU
In this section, the sentiment dictionary about danmaku is explored, mainly in terms of the pretreatment of the danmaku data and the expansion of the sentiment dictionary with emoticons.

1) DANMAKU DATA PREPROCESSING
Since Tencent Video has a relatively mature danmaku system with a large number of users (exceeding 1.1 billion) we selected danmaku data from Tencent Video for the database by testing the multiple types of online videos. Python technology with a request module was applied to obtain danmaku data. Compared to traditional online reviews, danmaku reviews are shorter and more colloquial and symbolic, often involving internet catchwords. Based on these characteristics, danmaku reviews are defined by two types of data, danmaku content and danmaku occurrence time (from the beginning of the video, in seconds). The data were cleaned according to the requirements of the study. By denoise processing, we removed the jamming danmaku that did not contain any emotional information Then, the Jieba Chinese word segmentation tool and a Catchwords dictionary were used for segmenting danmaku text and part-of-speech tagging. The rest of the words were kept using the word segmentation principle. Finally, we used a Stop words tool to remove the modal particles, punctuation marks and mathematical symbols, which have no actual meaning in sentiment analysis.
After the preprocessing of danmaku data, we built a corpus set A.

2) DANMAKU SENTIMENT DICTIONARY WITH EMOTICONS
The sentiment dictionary studied in this paper is mainly based on the Chinese emotional vocabulary ontology database of the Dalian University of Technology (DUT), which includes 27476 words (http://ir.dlut.edu.cn). According to sentiment dictionary of DUT, emotional words include ''like'', ''happiness'', ''surprise'', ''fear'', ''anger'', ''sadness'' and ''disgust'', the seven sentiment dimensions. The sentiments of ''like'' and ''happiness'' have positive sentiment polarity, while the sentiments of ''surprise'', ''fear'', ''anger'', ''sadness'' and ''disgust'' have negative sentiment polarity. The positive polarity intensity of emotional vocabulary can be divided into five levels of 1, 3, 5, 7, and 9. Similarly, negative polarity intensity values include -1, -3, -5, -7, and -9 to express the degree of negative intensity of emotional words. Because there exists a large amount of internet language in danmaku reviews, we adopted a Catchwords dictionary, which includes 733 network hot words and can be divided into seven sentiment categories, just as the sentiment dictionary of DUT, as a supplementary tool. In addition, considering the symbolic characteristic of danmaku reviews, it was necessary to add emoticons to sentiment dictionary. For this purpose, the emoticons with relatively obvious emotional tendencies were selected from corpus set A. Then, based on the seven-dimensional sentiment dictionary of the DUT, we made an artificial classification and gave weights to each of the emoticons, ranking the emoticons in order of importance. Finally, we summarized 161 emoticons and created an emoticon set. The seven sentiment dimensions of typical emoticons are shown in Table 1. The result is that we combined the emotional vocabulary ontology database of the DUT, the Catchwords dictionary and the emoticons set as a whole danmaku sentiment dictionary.

B. DANMAKU SENTIMENT CLASSIFICATION 1) FEATURE SELECTION BASED ON A SENTIMENT DICTIONARY
The diversification of Chinese online reviews and the fact that the length of sentences is unfixed are both challenges in feature extraction. Classical feature extraction methods include the following: a sentiment dictionary, n-gram and TF-IDF. An n-gram algorithm takes N as the fixed window length of segment text, and its extraction effect is better for text with the same length in feature words as N. However, the length of feature words in Chinese text is different, thus, using fixed windows for segmentation may cause semantic confusion. The sentiment dictionary can match all feature words in a text on the basis of traversing danmaku, thus avoiding the loss of feature words due to n-gram fixed-length cutting. The TF-IDF algorithm mainly relies on word frequency and weight for feature selection, ignoring the impact of text timeliness and text type on algorithm parameters (word frequency TF, inverse document frequency IDF). In comparison, the sentiment dictionary method is not affected by text source domain, text timeliness, or text type, which thus reduces the limitation of sentiment classification. Based on the above, the sentiment dictionary method was adopted to extract features in this study.
Danmaku is a kind of live commenting feature on network videos that relies on video content. The emotional expression of danmaku is complex and varied. The sentiment dimension of danmaku text is mostly reflected by sentiment words and some auxiliary elements, such as ''negative words'' and ''degree adverbs''. In this sense, the construction of a sentiment dictionary including comprehensive emotional words is a significant undertaking in danmaku sentiment analysis. The emotional vocabulary ontology database of the DUT, the Catchwords dictionary and the emoticons set constructed in this study jointly constitute a danmaku sentiment dictionary. For the efficiency and accuracy of identifying features of sentiment words, the sentiment dictionary-based approach is used in the sentiment classification. The main role of a sentiment dictionary for a segmented danmaku review is to extract the feature words and convert them to word vectors. After feature selection, a Naïve Bayes (NB) model was adopted to classify the sentiment of the danmaku comments.

2) SENTIMENT CLASSIFICATION BY NAÏVE BAYES
The Naïve Bayesian classifier is an optimization classifier model based on Bayesian statistics and the Bayesian network method. It mainly predicts the posterior probability that a danmaku sample belongs to a certain sentiment category according to the prior probability distribution and selects a sentiment category with the highest probability as the predicted sentiment category. Since the NB algorithm is stable and easy to operate, it is wildly used in sentiment classification. The mathematical model of the NB classifier can be expressed as follows: Define W (w 1 , w 2 , · · · , w n ) as the danmaku sample, and C(c 1 , c 2 , · · · , c 7 ) as the predicted sentiment category set of danmaku. The Mathematical model of NB of sentiment classifier can be expressed as: where w i , i ∈ [1, n] represents the feature words extracted from danmaku comments and c j , j ∈ [1, 7] represents the sentiment category of danmaku,c 1 = ''Like , c 2 = ''Happiness , c 3 = ''Anger , c 4 = ''Sadness , c 5 = ''Fear , c 6 = ''Digust , c 7 = ''Surprise . Since the feature words are independent of each other, the conditional probability P( W | c j ) is formulated as: According to the Bayesian formula, the sentiment classification equation (1) can be rewritten as: where P(c j ) is a prior probability of sentiment category c j and P( w i | c j ) is the posterior probability of the Bayesian model.
where n i=1 s w i , c j represents the number of sentiment words included in the training sample that belong to category c j . N is the total number of sentiment words contained in VOLUME 8, 2020 the whole training set,s w i , c j is the sum of frequency of sentiment word, and w i appears in each danmaku based on category c j .
In addition, to avoid a case in which the posterior probability P( w i | c j ) is equal to 0 during calculation, by using Laplace transform, the final posterior probability is gained from: where M is the number of sentiment words not repeated in the training set.

C. SENTIMENT VALUE CALCULATION OF DANMAKU
After completing Chinese word segmentation, stop word deletion and other process, the sentiment value calculation of danmaku was implemented. A danmaku sentence may contain some sentiment words, network words, or emoticons, the first two of which can be modified by negative words and degree adverbs. Here, define V is the sentiment value of a single case in a danmaku sentence, g denotes the weight of a sentiment word, h denotes the weight of a network word, θ denotes the weight of a degree adverb, and e denotes the weight of an emoticon. By analyzing emotional labels, sentiment scores for danmaku text can be determined as a combination of ''like'', ''happiness'', ''surprise'', ''fear'', ''anger'', ''sadness'' and ''disgust'' results. The following nine cases of weight calculation are considered: Case 1: When only sentiment words appear in a danmaku sentence, the calculating formula of sentiment value is: where l represents the number of sentiment words in a danmaku sentence. Case 2: When only emoticons appear in a danmaku sentence, the calculating formula of sentiment value is: where p represents the number of emoticons in a danmaku sentence. Case 3: When only network words appear in a danmaku sentence, the calculating formula of sentiment value is: where z represents the number of network words in a danmaku sentence. Case 4: When the sentiment word is modified by negative words, the calculating formula of sentiment value is: where r i , i = 1, 2, . . . , l represents the number of negative words appearing in front of the i-th sentiment word. Similarly, we conduct case 5 for the sentiment calculation of network words modified by negative words.

Case 5:
When the network word is modified by negative words, the calculating formula of sentiment value is: (11) where r j , j = 1, 2, . . . , z represents the number of negative words appearing in front of the j-th network word.
Case 6: When the sentiment word is modified by degree adverbs, the calculating formula of sentiment value is: Case 7: When the network word is modified by degree adverbs, the calculating formula of sentiment value is: Case 8: When the sentiment word is modified by both negative words and degree adverbs, the calculating formula of sentiment value is: where, (−1) r i q 1 ui=1 θ ui g i implies that the i-th sentiment word is modified by r i negative words and q i degree adverbs. Case 9: When the network word is modified by both negative words and degree adverbs, the calculating formula of sentiment value is: where (−1) r j p j uj=1 θ uj h j implies that the j-th network word is modified by r j negative words and p j degree adverbs. Consequently, in a case in which a danmaku sentence contains all of the above cases, the calculating formula of sentiment value can be expressed as:  the winner receives a prize. Using Python, 54 1841 danmaku reviews of ''Keep Running'' season 2 and season 4 were collected. After data preprocessing, 301052 danmaku reviews were left for the sentiment analysis, and analysis algorithms were implemented by Python. Based on the construction of the danmaku sentiment dictionary and sentiment classification described in Section 3, the sentiment analysis of danmaku reviews for ''Keep Running'' season 2 and season 4 will be carried out.

B. RESULTS OF SENTIMENT ANALYSIS 1) VISUALIZATION OF DANMAKU DATA
Based on the number of danmaku reviews with multidimensional sentiments, a radar map was drawn. Fig. 2 displays the distribution of danmaku reviews in the seven sentiment categories, which provides a basis for the further analysis of the reviews' emotional tendencies. In Fig. 2, each circular ring reflects the percentage of each sentiment category, for instance, ''like'' was at 50%, ''happiness'' at 13%, ''disgust'' at 19%, ''sadness'' at 8.5%, ''anger'' at 3.5%, ''fear'' at 3%, and ''surprise'' at 3%. The radar map shows that positive sentiments are most prominent, meaning there are more danmaku reviews expressing ''like'' and ''happiness'' in ''Keep Running'' season 2 and season 4. As seen in Fig. 2, the percentages of some negative sentiments, such as ''sadness'', ''anger'' and ''fear'', are relatively small, but ''disgust'' still sits at 19% and ''sadness'' at 8.5%. Most often, these negative affectivities come from ''black picks'' (anti-fans), who just want to express dislike for a particular star or share a complaint about the result of sports game. Generally, danmaku reviews express viewers' personal emotions and opinions with regard to network videos, which also reflect the popularity of certain types of videos from another side. A tag cloud is a visual description of the keywords used to summarize user-contributed tags or text content. The purpose of word clouds in this study is to visually represent danmaku data in a visually coded way to show the, the relative importance of the danmaku sentiments of persons viewing video various programs. Sentiment words appearing more frequently in danmaku reviews are represented with a larger font and are highlighted with different font colors. According to these principles, we selected 367 sentiment words that were most commonly used in danmaku reviews for ''Keep Running'' season 2 and season 4 to draw the word cloud graph shown in Fig. 3. It can be seen that the more prominent emotional labels are high frequency words, such as LOL, like, come on, awesome, great, nice, funny and speechless. After understanding the sentiment distribution of danmaku reviews and identifying the high-frequency sentiment words that appear in danmaku reviews for ''Keep Running'' season 2 and season 4, we had a general idea of viewers' feelings toward the sports reality show. That is, for most viewers, the sports reality show is entertaining and makes people feel relaxed and happy.
Because of the real-time nature of danmaku reviews, danmaku sentiments are likely to depend on time and the number of danmaku comments. Fig. 4 (a) and (b) reflect the dynamic changes in the number of danmaku comments at each timestamp. The fold line denotes the number of danmaku reviews per 30 s. As we can see from the graphs, the number of danmaku reviews changes significantly as the video program progresses. Since danmaku data is a type of unstructured moment-to-moment data, it is related to a video's content and timing. Take ''Keep Running'' season 2, episode 10 and season 4, episode 8 as examples: as seen in Fig. 4 (a), in the time period 0 min∼94 min, the number of danmaku comments is relatively stable at a range of 109∼127 comments, while in the time period 94 min∼101 min, the content changes and only one comment appears. This because at the end of program, there is a game summary presented by the host, which is less attractive to viewers. By contrast, in Fig. 4 (b), the number of danmaku reviews decreases correspondingly with the video playback time. There are occasional peaks and troughs in the fold line. For instance, in the time periods of 10:30, 16:30, 29:30, 56:00 and 93:30, the number of danmaku reviews hits its peak. The plots of these time points might resonate with viewers, who post danmaku reviews to express what they see and feel. However, in the time periods of 33min∼34min, 37min∼38min, 44min∼47min, and 98min∼104min, troughs appear in the fold line, as at these points, the content is flat and unappealing, and viewers are thus reluctant to create danmaku reviews here. show that, generally, the absolute value of positive sentiments is significantly higher than that of negative sentiments, while some exceptions exist, such as the cases marked with circles, where the negative sentiment value is greater than the positive value in the same time period. In comparing Fig. 6(a) and (b), the distribution of two-polarity sentiment value is relatively homogenous in Fig. 6(a). However, the distribution of positive sentiment values varies more than that of negative sentiment values. This section presents some experimental results from an analysis meant to examine the performance of several sentiment classification algorithms with real-world data. To evaluate the effectiveness of the proposed Sentiment Dictionary-Naïve Bayes (SD-NB) method for the danmaku sentiment analysis task, we carried out comparative experiments of SD-NB with N-gram-NB, N-gram-SVM and TextCNN. For this, 65000 danmaku reviews were randomly selected from the 301052 danmaku data from ''Keep Running'' season 2 and season 4. After manual annotation (32000 positive and 33000 negative) and cross validation, we obtained a dataset named KR(s2, s4) for classifier evaluation. Then, we randomly split KR(s2, s4) into two subsets, 70% for the training set and 30% for the test set, keeping the proportion of classes the same. In the following, dataset KR(s2, s4) is used to evaluate the performance of SD-NB, N-gram-NB, N-gram-SVM and TextCNN. SD-NB: A linear classifier that combines danmaku sentiment dictionary with the Naïve Bayes method. We constructed a danmaku sentiment dictionary, then applied it to extract the feature words and convert them into word vectors. After feature selection, a Naïve Bayes (NB) model was adopted to classify the sentiment of the danmaku comments.
N-gram-NB: A standard representation is used in text classification tasks. Documents are represented in a space of unordered list of terms (n-grams) as word vectors. We applied n-gram model for feature selection from danmaku text, and Naïve Bayes (NB) algorithm for sentiment classification. Considering the characteristics of Chinese language, in order to obtain a good classification effect, we took n = 2 in the experiment [54].
N-gram-SVM: A popular machine learning method for classifying linear problems. After completing word segmentation, stop word deletion and other process, we utilized n-gram model to extract features. In the experiment, the word vector dimension was set at 300, and the SVM algorithm was used for training and fitting.
TextCNN: A CNN model for classification with word embeddings [55]- [56]. It is a useful deep learning algorithm for short text classification. After the preprocessing of the danmaku data, Word2Vec was used to convert the danmaku comments into sentiment word vectors, where Sikp-gram was the feature extraction method. In the experiment, we proposed a one-layer CNN model, which involved 256 convolution kernels with a size of 5 and 128 neurons in the full connective layer. The TextCNN model trained 64 danmaku texts each time and ran for 10 iterations, with learning rate of 0.001.

2) PERFORMANCE OF THE EXPERIMENTS
We followed four evaluation measures, i.e., precision, recall, F1 score, and accuracy, which are widely used in information retrieval [57] for text sentiment classification tasks (see Eq. (17) and Eq. (18)).
VOLUME 8, 2020 where TP is the number of true positives, FP is the number of false positives, FN is the number of false negatives, and TN is the number of true negatives. Table 2 shows the performance of three sentiment classification algorithms on KR(s2, s4) dataset. Overall, the quantitative results across all four evaluation criteria (precision, recall, F1 score and accuracy) indicated that our proposed SD-NB outperforms N-gram-NB, N-gram-SVM and TextCNN. In particular, the accuracy of SD-NB can achieve 0.882, making the relative improvement over N-gram-NB 17.6%, N-gram-SVM 20.0% and that over TextCNN 14.8%. The results of precision, recall, F1 score and accuracy were stable in the SD-NB model. The F1 scores of SD-NB were the highest among all classification algorithms: 82.3% for the positive class and 93.6% for the negative class. Besides, in SD-NB algorithm, the difference between precision and recall was relatively small. While for N-gram-NB, N-gram-SVM and TextCNN, the quantitative results were not stable. Especially in N-gram-NB and N-gram-SVM models, the precision and recall of the positive class were quite different from those of the negative class.
According to the evaluation results, the performance advantages of SD-NB versus N-gram-NB, N-gram-SVM and TextCNN in Table 2 are significant. Hence, the sentiment dictionary combined with Naïve Bayes can be considered the appropriate algorithm for danmaku sentiment classification.

V. CONCLUSION
The booming development of network media-sharing has produced a large amount of video data. For application to danmaku videos, this paper presents a method using a sentiment dictionary and Naïve Bayes (SD-NB) for the sentiment analysis of danmaku reviews. This method is greatly helpful for supervising the overall emotional orientation of a danmaku video and predicting its popularity. Through the processes of extracting emotional information on danmaku videos, classifying sentiment and visualizing data, the time distribution of seven sentiment dimensions and sentiment analysis results can be obtained, based on which, we may draw some conclusions and inspiration as follows.
We constructed a danmaku sentiment dictionary to expand the existing sentiment dictionaries of the DUT and Catchwork with the 161 main emoticons provided by the danmaku video platform. Based on the danmaku sentiment dictionary and Naïve Bayes classification, the time distribution of the seven sentiment dimensions and two-polarity sentiment values are studied in this paper. Applying sentiment analysis, we obtain the result that both danmaku review volume and sentiment value are dynamic and change with time and video content. Because the length of feature words in Chinese text is different, using fixed windows for segmentation (n-gram) may cause semantic confusion. The sentiment dictionary can match all feature words in a text on the basis of traversing danmaku, thus avoiding the loss of feature words due to n-gram fixed-length cutting. In comparative experiments, the performance of proposed SD-NB is better than that of N-gram-NB and N-gram-SVM. The study of the characteristics and sentiment expression of danmaku data will help us to understand video content and thus further explore the potential relationship between danmaku reviews and video content. This can provide a more effective solution for the selection of audience groups in video production. Meanwhile, through a large number of danmaku-based user interaction data, video producers can accurately depict the advantages and disadvantages of each video clip, and then adjust their subsequent video production.
Based on a sentiment analysis of danmaku comments, a new approach to video retrieval can be established to meet personalized and diversified retrieval requirements. In some sense, the introduction of the danmaku function to network videos can greatly improve users' activity and enrich their interactive behavior data. Because of the real-time relevance of danmaku reviews, we may obtain online comments on specific video clips. In such a way, user preferences can be described more accurately and can then be utilized to optimize the layout of a video website, develop advertising strategies, carry out personalized video recommendations and provide other intelligent services.