Sentiment Analysis of Weibo Comments Based on Graph Neural Network

Weibo is one of the most important online social platforms. Currently, user comments are increasing rapidly, which makes data management difficult. Comments show the non-standardized and colloquial form of expression. Traditional sentiment analysis techniques are no longer applicable to unspecified sentence analysis tasks. To mitigate overreliance on text sequences, ignoring syntactic structure, and the poor interpretability of feature space that are typical of traditional classification models, a sentiment classification model based on a graph neural network (GNN) is developed in this study. For each comment text, the dependency syntax is used to construct the semantic graph of the short text. Aiming at the heterogeneity of the semantic graph, the spatial domain graph filter is designed for feature extraction. Concurrently, long short-term memory (LSTM) is used as a state updater to filter node noise. In this method, a graph neural network is used as a semantic parser to encode the syntactic dependency tree, which can extract the semantic and syntactic features of sentences concurrently. Experimental results show that GNN-LSTM has achieved superior performance in the Weibo comments dataset by achieving 95.25% accuracy and 95.22% F1 score.


I. INTRODUCTION
Sentiment analysis is one of the primary tasks of natural language processing and requires identifying the trend of sentiment of users toward a given piece of text. Sentiment analysis has become an important analysis tool for the many user comments that are generated in online social platforms. Sentiment analysis algorithms based on deep learning are currently popular, and the deep learning algorithms proposed for different levels of tasks can achieve good results [1]. Recently, user comments have been rapidly increasing on online social platforms. Mining thematic views on massive texts and identifying their sentiment tendencies can provide a clear understanding of users' opinions for platforms, which helps platforms interact with users and optimize promotion strategies; this process also helps governments understand public opinion trends and make management decisions. Therefore, the task of sentiment analysis for users' short comments has become urgent. However, the widespread use of social platforms has accelerated the spoken expression of Internet phrases, and traditional sentiment analysis techniques are no The associate editor coordinating the review of this manuscript and approving it for publication was Arianna Dulizia .
longer applicable to the current non-standardized sentence analysis tasks. Traditional classifiers rely on text sequence representation and ignore syntactic components, and thereby construct a poorly interpretable feature space with Internet phrases. Models thus fail to achieve better optimal results when extracting continuous features, reducing classification performance.
To address this problem, this study develops and tests a text sentiment analysis method based on graph neural network and long short-term memory (GNN-LSTM). The semantic rules and structural dependencies of short texts are effectively preserved through syntactic analysis, and a GNN semantic parser is used to complete unstructured feature extraction. The fusion method applies a syntactic tree and GNN to sentiment analysis, which provides some theoretical and practical value when improving the performance of short text sentiment classification tasks.
The primary contributions and innovation of this paper are as follows: 1) We construct short text semantic graphs as the direct input to the graph neural network using syntactic dependency trees. The text classification task is thereby transformed into the graph classification task. Based on our knowledge, this VOLUME 10, 2022 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ method has rarely been applied to sentiment analysis, particularly for Chinese comment analysis tasks.
2) We propose a GNN-based classification model for sentiment analysis of Weibo comments. This model explores the effectiveness of the GNN semantic parser and LSTM update and constructs an interpretable feature space. We study the classification performance of the model on a large set of Weibo comments. Additionally, the optimization strategy to mitigate oversmoothing is explored.
3) We analyze the performance of the GNN-LSTM model on a Weibo comment dataset. In addition, comparative experimental results show that GNN-LSTM also achieves good performance when oriented to comments from other platforms, demonstrating that the model has good generalizability and robustness on open online comment areas.

II. RELATED WORK A. THE SENTIMENT ANALYSIS ALGORITHMS BASED ON DEEP LEARNING
Machine learning [2], [3] has more objective results compared to the construction of sentiment dictionaries [4], reduces the need for human resources, and solves problems such as difficulties in updating the corpus. However, it is more demanding when constructing datasets.
To address the shortcomings of machine learning algorithms, the nonlinear hierarchical neural network is used to approximate the complex function representation and learn the deep features of the corpus. Online texts can thus be processed quickly and accurately by neural networks. Therefore, deep learning methods based on neural networks are the primary research direction of current sentiment analysis [5]- [8]. As time series models, Recurrent neural networks (RNNs) are widely used in natural language processing. The unique structure of the model, which is suitable for processing context dependencies, improves classification performance over traditional machine learning algorithms [9]. However, the sequential model easily generates a large amount of redundant information and results in gradient explosion or disappearance problems. Researchers have introduced the threshold mechanism and memory unit based on RNN and proposed the LSTM model to mitigate these issues. By controlling the parameters that determine both retained and forgotten information, longer distance dependencies can be better captured. In the literature [10]- [12], the sentiment classification method based on LSTM has been proposed, and experimental results show that the LSTM model can achieve a classification accuracy of more than 90% by itself, which markedly exceeds the performance of the support vector machine (SVM). However, the LSTM model can only encode one-way sequence information. To capture bidirectional semantic dependencies, the bidirectional long short-term memory (Bi-LSTM) method was developed by combining forward and backward LSTMs to obtain the splicing vector to obtain all the information on the context. In the literature [13], [14], word-level and sentence-level features are extracted through Bi-LSTM to synthesize local sentiment in a recursive order to obtain sentiment labels of the entire text, which can capture the feature differences between sentiment polarities more accurately. However, due to the inadequacy of Bi-LSTM in capturing local semantic features, researchers have introduced a combination of Convolutional Neural Network (CNN) [15] and attention mechanisms [16]- [18] to facilitate better extraction of effective focused information [19]. In the literature [20], CNNs have been used to receive parallelized input information and combine them with multiple attention mechanisms, which effectively compensate for the overreliance on content-level attention mechanisms and identify the sentiment polarity of different targets. The literature [21]- [25] also combines improved Bi-LSTM and CNN models to capture long-term dependencies with CNNs using RNNs, obtaining better classification results.
These studies perform well on text sentiment classification tasks. However, the models only rely on sequence representations from the perspective of Euclidean data structures, ignoring the dependencies between sentence components. Thus, the constructed feature spaces are not interpretable. In response to the black-box characteristics of neural networks, new neural network models have been proposed [26], but the interpretability of each layer remains difficult to manage during model construction. To address these problems in neural network models, researchers have introduced graph neural networks (GNNs) as syntactic structure encoders in semantic analysis, providing a new research perspective to improve the performance of text sentiment analysis.

B. GRAPH NEURAL NETWORK
The application of GNNs to graph classification tasks [27] has inspired researchers. The GNN can be extended to text classification tasks by transforming texts into graphs through dependency syntax and then introducing a graph neural network as novel semantic encoder. The GNN can effectively manage complex relational structures of text and preserve global information in feature embeddings. Diego and Ivan proposed a semantic role annotation algorithm based on a graph convolutional network (GCN) that uses a GCN to encode syntactic dependency trees to produce potential feature representations of words in sentences. This study demonstrates the utility of GCN in NLP for the first time [28]. Liang Yao and Chengsheng applied a GCN to text classification tasks and constructed a corpus as a large graph containing word nodes and document nodes using GCN to capture higher-order neighborhood information. The text classification problem was thus transformed into a node classification problem, and better classification performance was obtained even with only a few annotations [29]. Due to the shortcomings of existing neural network models that ignore syntactic representation, T.H. Nguyen proposed a GCN event detection model based on syntactic dependency trees and a new pooling method to achieve optimal results [30]. For this type of problem, Shucheng Li and Lingfei Wu also proposed a neural semantic parser called Graph2Tree based on the graph neural  network, which consisted of a graph encoder and hierarchical tree parser, and achieved remarkable results [31].
These studies effectively demonstrate the feasibility of GNNs as semantic parsers applied to NLP tasks. Currently, based on previous studies, some researchers have extended the combination of dependency syntax and GNN to sentiment analysis [32]. Shuncheng Yang and Yan Li combined topic-specific targets and proposed the Weibo stance detection method based on GCN and Bi-LSTM, which effectively improves detection accuracy [33]. However, existing research based on GNN of sentiment analysis still has a wide space for exploration and must be extended, particularly with regard to the analysis of Chinese comments.
Although the GNN as a semantic parser optimizes the deficiency in the sequence model, there are still some problems to be optimized, including the following: 1) The feature initialization of some models still depends on the sequence vector. The dependent syntax is only introduced as external knowledge to expand the feature space, but the dependency relations between sentence components cannot be directly extracted and used; 2) For the characteristics of personalized and colloquial online language, the sentence-level syntactic relationship tree has not been established. The lack of supplementation for other feature information affects classification performance; 3) The introduction of graph data typically includes a lot of noise, which may adversely affect model performance.
To address these issues, a sentiment analysis model based on GNN-LSTM is proposed. The statement of this method compared with other methods is described in Table 1. The advantage of the proposed method is that it can simultaneously extract semantic features and syntactic features to construct an interpretable feature space. First, the short review is transformed into the syntactic tree. Distinct from existing methods, we process each comment individually and transform each sentence into a semantic graph with explicit relationships instead of constructing the entire corpus as a large connected graph. Then, graph convolution operations are performed on the semantic graph to extract abstract semantic features. Finally, the feature representation of the entire graph is read out and used for sentiment classification of the short texts.

III. METHOD FOR SENTIMENT ANALYSIS BASED ON GNN-LSTM
The sentiment classification model proposed in this paper is shown in Fig. 1 and contains four primary modules: preprocessing, syntactic analysis, semantic graph construction, and spatial graph filter construction.
The specific implementation steps are as follows: 1) First, the dataset is preprocessed, preserving valid characters.
2) Second, syntactic analysis is performed, including the three steps of word segmentation, tagging and dependency parsing.
3) Word2Vec is then used to embed subword nodes and dependent edges, and the semantic graph is constructed. 4) Then, the spatial graph filter is designed, the GNN-LSTM model is constructed, and the aggregation of features and updated states are completed.
5) The task of sentiment classification uses the softmax classifier. Finally, the model is tested to complete the performance evaluation.

A. SEMANTIC GRAPH CONSTRUCTION
Syntactic analysis analyzes grammatical rules and sentence constituents for a corpus. Phrase structure syntax and dependent syntax are the grammatical systems that are commonly used in a corpus. Dependent syntax is a theory of sentence syntax created by the French linguist L. Tesniere. Dependency syntax can transform short texts into syntactic dependency trees and simplify sentence expressions. The syntactic analysis module of the Language Technology Platform (LTP) designed by the Harbin Institute of Technology uses the dependency syntax system, which can manage 14 types of dependency relations shown in Table 2. LTP can also perform sentence component annotation and relation annotation. The construction of the semantic graph applies the dependency syntax by labeling the dominant subordination relationships between sentence components and transforming a short text into a tree topology. The nodes of the semantic graph are each segmented, and the root node is the subject verb, which is not controlled by other components and dominates all the remaining components. If there is a dependency relationship between the components, the edge is created between two nodes, and the relationship attribute is given. For example, for ''LPT (LTP provides a series of Chinese natural language processing tools)'', the annotated syntactic dependencies and the transformed tree topology are shown in Fig. 2. The process of converting a text to a semantic graph is as follows: 1) The special characters are filtered, and textual information is retained.
2) The PYLTP module is introduced, completing word segmentation, tagging and dependent syntactic analysis.
3) After obtaining the segmentation and syntactic relations, semantic graph construction is performed. The segmentation is considered as nodes, and dependency edges are established. The attributes of edges are assigned to 14 dependencies. Algorithm 1 describes this process in detail. 4) Word2Vec is used to embed nodes and edge attributes. The sentence-level sentiment classification task is transformed into the graph classification task by converting a short text into a semantic graph.

B. GRAPH NEURAL NETWORK
A GNN is a connectedness model that captures graph relationships through message passing between nodes. Compared with traditional neural networks, GNNs can aggregate information from arbitrarily deep adjacent nodes around a node [34]. A GNN can be considered to be an extension of CNN from Euclidean data to non-Euclidean data. Using two methods (a spatial domain filter and a spectral domain filter), the GNN can effectively extracts the spatial features of topological graphs and extend the CNN to manage non-Euclidean for j=1 to k and j !=i do 5: if W j → W i then 6: Connect node i and node j → e i,j 7: Connect node j and node i → e j,i 10: end if 12: end for 13: end for 14: return G data. The spatial domain filter is more intuitive than the spectral domain filter. The overall features of the graph can be obtained by iteratively aggregating the adjacent features of each vertex. Also, the spectral domain filter implements convolution operations on topological graphs based on spectrum theory and computes the eigenvectors of graphs with the help of the graph Laplacian matrix to complete normalization. In practical applications, the spectral domain filter is restricted to fixed connected graph processing. Therefore, the construction of a semantic graph is required. In contrast, the graph convolution operation of the spatial domain filter is more flexible and has a wider range of application scenarios. Through the graph convolution operation, the GNN can obtain all node embedding representations. The embedding representation of the entire graph can be obtained by a pooling operation called readout. The graph classification task can be accomplished by graph-level representation [35].
The graph filter is designed to aggregate the features of nodes and edges in the semantic graph and extract semantic and structural features from a short text. In this paper, syntactic trees are constructed independently for each data, with different features such as the number of nodes and edges. They are heterogeneous graphs with more complex structures and are suitable for flexible spatial domain graph convolution operations. Therefore, the spatial domain graph filter is designed to complete the graph convolution on the semantic graph.

C. MPNN FRAMEWORK
A Message Passing Neural Net (MPNN) is a formal framework of spatial domain convolution. An MPNN is not affected by graph isomorphism and places the complex and intractable spectral domain convolution under another intuitive and common methodological perspective, which increases the flexibility of application scenarios. The MPNN summarizes the spatial graph convolution into two steps, as shown in Fig. 3: message passing and state updating [36].
where M t is the message delivery function, and N (i) is the source node, and N (j) is the adjacent nodes of N (i) . t is the time step, and ρ is the method of aggregating, such as Max, Add and Average.
where U t is the update function, and U t and M t are both differentiable functions. The readout is calculated as follows: R-functions are differentiable functions that are used to aggregate node states and must be independent of node alignment. G is the graph data, and X i is the iterative state of N i .

D. SPATIAL GRAPH FILTER
For heterogeneous semantic graphs, the graph convolution operation on the spatial domain is used. A spatail domain graph filter based on the MPNN framework is designed. The graph filter of the GNN-LSTM model is set as follows: where X t is the state of the node at t iterations, and L t is the graph filter that aggregates t-th-order features. The graph filter still retains the parameter sharing property of convolution in the non-Euclidean space. The sharing mechanism of graph convolution parameters is shown in Fig. 4.
To capture the semantic information and dependencies of nodes, 14 dependency encodings are introduced as inputs to the graph filter as edge feature weights. In addition, the state of the source node depends on the states of K-group adjacent nodes, and the feature transfers of K-order adjacent nodes can be completed by K iterations. In general, a better effect of feature extraction can be achieved when the K value is taken from 1 to 2. In this study, after the comparison of experimental results, K is set equal to 3.
Due to the complex graph structure, the syntactic tree is introduced with a large amount of noise information, which affects model performance. To address this issue, the filter must retain the serialized feature analysis to optimize the model performance.
Eq. (1) and (2) are thus rewritten to propose the spatial domain graph filter of GNN-LSTM: where e ij is the weight vector of the dependent edge between N (i) and N (j) ; ij represents the existence of connectivity between N (i) and N (j) ; ''Mean'' indicates that the method of feature aggregation is to take the mean of adjacent features; the || operator finds the Euclidean distance; and φ t is the multilayer perceptron.

1) MESSAGE PASSING
Aggregation is performed by finding the mean value of all adjacent features. The feature extraction operation uses Euclidean distances to calculate the feature differences between the source and target nodes. The attribute (dependency) of the edge between the two nodes is concatenated with the distance vector and fed into the function of message passing that obtains the delivery message at moment t.

2) STATUS UPDATE
Because the state update updates the node features from moments t to t+1, there is a temporal relationship. There is also semantic noise in the features of aggregated nodes that must be filtered. Therefore, the LSTM model is used as the update algorithm. LSTM as a sequential model has the output at moment t: The current output depends on the previous moment state and the current cell state, which can be accomplished by updating iterations of the state at each moment. Combining LSTM with GNN can perform a serialization operation to mitigate noise and preserve the interpretability of semantics and syntax while capturing local features and aggregating global features on the graph. Thus, the two models are complementary.

3) READOUT
The number of nodes and edges in the semantic graph transformed by each text is different; thus, the graph structure is heterogeneous, which requires the adjacency matrix to be concatenated to achieve parallelization of mini-batch during processing. The operation performed is: In each batch, the index of the graph is added to each node to distinguish the nodes in each graph: Readout operations on mini-batch can be performed by global pooling layers (GPLs). Pooling methods include global_max_pool, global_add_pool and global_mean_pool, etc. The feature vector of each semantic graph is computed by indices in batch. In this paper, we choose the readout function: To reduce the readout loss, both global_add_pool and global_mean_pool are used to read out the graph features. The global_add_pool is computed as equation (10), and global_mean_pool is computed as equation (11).

E. GNN-LSTM MODEL CONSTRUCTION
The structure of the GNN-LSTM model for sentiment classification by extracting semantic and structural features is shown in Fig. 5. Assuming that the word segmentation of short text isembedding word vectors is T = where φ A is the soft-attention mechanism that assigns the weight to the feature differences of the nodes; f is the sigmoid activation function; is the concatenation of node features and edge features; and Dφ t is a linear transformation and ReLU activation to generalize the model. Finally, the mean value is taken for features of all adjacent nodes. For the constructed semantic graph to reduce redundant features, no self-connected edges are added to the subword nodes. The nodes' own features are passed instead in the update stage?
Combining the node's own feature with the adjacency feature as input at moment t, the LSTM model completes linear transformation and activation to obtain the state of the source node with adjacency features at moment t+1. This step completes the state update.
The graph convolution operation is then iterated three times. To stabilize the model convergence rate and stability, LayerNorm calculations are performed as Eq. (14) after each convolution layer to ensure that the features converge to the same distribution before each pass [37]: The mean and standard deviations of all nodes and channels in a mini-batch are calculated and then activated by ReLU. Finally, the aggregated 3rd-order adjacency feature is read out by the global pool algorithm and fed into the softmax function for classification.

A. EXPERIMENTAL DATA SET
1) The Weibo_senti_100k dataset contains more than 120,000 Weibo comments, with approximately 60,000 positive and 60,000 negative comments. From this dataset, the comments are randomly divided into 70% as the training set and 15% as the test set (WB18K); then, the data are randomly divided into 70% as the training set and 10% as the test set (WB12K). Tests are performed on two test sets. The divisions of the dataset and examples are shown in Table 3.
2) The online_shopping_10_cats (Online shop) dataset has approximately 60,000 online shopping reviews containing 10 product categories, with more than 30,000 positive and negative reviews each. The comments of 10 product categories are combined. Seventy percent of the comments are randomly divided into the training set, and 15% of the comments are randomly divided into the test set. The divisions of the dataset and examples are shown in Table 4.
3) The book review dataset (Book): it has 20,000 book reviews, approximately 10,000 positive comments and 10,000 negative comments. The comments are randomly divided into 70% as the training set and 15% as test set. The divisions of the dataset and examples are shown in Table 5.

B. EVALUATION CRITERIA
To validate the model performance, accuracy, cross entropy loss function (Loss) and F1-score are selected as evaluation VOLUME 10, 2022 where TP (true positive) indicates the positive sentiment with correct predictions, FP (false positive) indicates the positive sentiment with incorrect predictions, FN (false negative) indicates the negative sentiment with incorrect predictions, n is the total sample size, x is a specific sample data point, y is the true label, and a is the predicted output.

C. PARAMETER SETTING
The Adam optimizer and cross-entropy loss function are used for training, and the experimental results are compared after several parameter adjustments. Finally, the parameters are set to Table 6, and the sentiment classification model can determine the optimal effect. During the experiment, we rearrange the dataset and randomly divide it into the training set, validation set and test set. The experiment is repeated three times, and we report the mean of the test results as the evaluation result. Based on the parameters described above, 80% of the data from the Weibo comment dataset containing approximately 120,000 short texts are divided as the training set. Ten percent of the data are divided into the validation set, and 10% of the data are divided into the test set for testing. The validation effect of the first 100 epochs on the validation set is shown in Fig. 6.
The training curves in Fig. 6 show that the GNN-LSTM model converges quickly and achieves a high prediction accuracy. During the training process, the minimum loss obtained on the validation set is set to Best-Loss, and the corresponding parameters are saved for testing after the model converges.
The results are obtained on the test set with an accuracy of 0.9485, an F1 score of 0.9480, and a loss of 0.1394.

D. COMPLEXITY AND STABILITY
We evaluate the complexity of the proposed method using 2 parameters: 1) Spatial complexity (the number of model parameters) Table 7 shows the primary working modules of the model: graph convolution layer and the fully connected layer. The primary composition of the network layers and the output shapes are shown in table. After decomposing the model modules, we completed the statistics of the model parameters and calculated the real disk space occupied by the model. The calculation results are shown in Table 8. Additionally, we calculate the running time, taking the mean value to train one graph. Results show that the number of training parameters is approximately 1.0987MB, and the running time is approximately 0.03s, which achieves relatively balanced consumption in terms of space and time.

2) Time complexity (the number of model operations)
The proposed model is implemented based on the spatial domain graph filter. The computational steps are broken down into two steps: transfer and update. Feature passing is completed by a mean function including features of adjacent nodes and edges. Because the semantic graph is sparse, the index lists of nodes and edges are used for storage in real calculation. Assuming that the semantic graph has N nodes and E edges, then there are at most N-1 adjacent nodes and dependent edges of a source node because there are pointing relations of dependent syntax. Therefore, the time complexity of the transfer stage is: The update uses LSTM as the updater, and the time complexity of LSTM is noted as the time complexity of the update stage. The overall complexity is obtained by adding the complexities of two computational stages: (19) where N is the number of nodes in the graph; D is the vector computation; D V , D E , and D S are the vector computation of node features, edge features, and adjacent features, respectively.
From Eq. (19), the time complexity of the proposed model is: where n is the number of nodes, and d is the vector length. Based on this inference, we estimated the model computation in Table 8, and evaluated the model with FLOPs, which obtained a value of approximately 1.807 × 108.
According to this complexity analysis and the training status in Figure 6, the proposed model has a moderate number of parameters and a reasonable trainable space. The proposed model also achieves a high computational efficiency and generalizability. Therefore, the model has high stability and robustness.

E. OVER-SMOOTHING OPTIMIZATION
The GNN aggregates adjacent features through message passing and obtains global graph features by continuous iteration. This aggregation will eventually cause the node states to converge. Even if the graph convolution layer is added to expand the perceptual range, no more effective features can be extracted to improve model performance due to oversmoothing that is common to GNNs.
Mitigating this issue optimally requires adding noise and aggregating differentiated features to improve model generalizability during message passing. Therefore, the strategies of node-feature loss and edge-feature loss are used to mitigate oversmoothing optimally using the following strategies: 1) Strategy of node-features loss: node features extracted from each convolution layer are input to the dropout layer during message passing, losing 20% of features. This operation enhances the data, and expands the difference in node features passed by each layer.
2) Strategy of edge-features loss: all edge attributes are randomly deactivated by 50% in the input stage. This strategy performs feature transformations for 14 fixed dependencies to increase feature diversity and distinguish edge states. The features are then input to the graph filter along with the node features for message passing.
Validation results with and without optimization are shown in Fig. 7.
The curves in Fig. 7 show that the model exhibits marked overfitting when the optimization strategy is not used. After applying the optimization strategies, oversmoothing is reduced, and generalizability is enhanced. Overfitting is thus effectively prevented.  Test results after applying the optimization strategies are shown in Table 9. The test results after adopting strategy 1 achieved an accuracy of 0.9500, an F1 score of 0.9495, and a loss of 0.1320. Compared to the model without optimization, performance improved by approximately 0.15%. With strategy 2, the test results achieve an accuracy of 0.9525, an F1 score of 0.9522, and a loss of 0.1312. Model performance is thus improved.

F. ABLATION EXPERIMENTS 1) EFFECT OF NETWORK LAYERS AND UPDATE
Two experiments were performed as follows: Model performance was compared when the convolution layer K was set to 1, 2 and 3.
The model performances of the GNN and GNN-LSTM were compared to demonstrate the effectiveness of the LSTM updater.
Comparative results are shown in Table 10, and the optimal classification performance is obtained when the number of layers is 3. As the state updater, LSTM can effectively remove the feature noise of the graph and optimize the effect of classification.

2) EFFECT OF NORMALIZATION
Comparison experiments were performed as follows: The functions of normalization as BatchNorm (BN), Lay-erNorm (LN), GraphNorm (GN) and non-normalization were used in the normalization layers to compare classification performances.
Comparative results are shown in Table 11, and the experiments show that LN can effectively prevent overfitting and improve generalizability.

3) EFFECT OF AGGREGATION SCHEME
Comparison experiments were performed as follows: When implementing GNNConv based on the MPNN framework, the aggregation strategies of adding (ADD), averaging (MEAN), and maximizing (MAX) are used to explore the impact of aggregation methods on performance.
Experimental results are shown in Table 12 and indicate that the adding strategy retains feature integrity while retaining a lot of noise. The maximizing strategy may lose some effective information. In contrast, the aggregation method of averaging can effectively obtain the adjacent features and has good performance.

G. COMPARISON OF RELATED WORK
The benchmark methods for the comparison experiments are described as follows.
2) Bidirectional Long Short-Term Memory (BiLSTM) [39]: The model consists of a forward LSTM and a backward LSTM spliced together. Forward and backward hidden vectors are concatenated for sentiment classification tasks.
3) TextING [40]: This algorithm constructs a text graph by unique words as vertices and co-occurrence relations between words as edges. Feature embeddings merged with their own features are obtained by gated GNN for updating. Finally, the graph-level feature is read out. 4) TextGCN [29]: It constructs a heterogeneous graph for the entire corpus, with deduplicated words and documents as nodes. Edges are constructed between documents and words, and words and words. Document features and lexical features VOLUME 10, 2022 are embedded using TextGCN. The node features of the last layer are used for label prediction. 5) BiLSTM-GCN [33]: Word vectors are first input into the BiLSTM model for encoding. The resulting hidden vectors are used as the initialization input to the GCN. Dependencies of distant nodes are captured by the GCN. Finally, the representation of text features is obtained by the global pooling operation for sentiment classification.
The performance of these models is compared, and results are shown in Tables 13 and 14. ''WB18K'' denotes the test set with 1,8000 Weibo comments; ''WB12K'' denotes the test set with 1,2000 Weibo comments; ''Online shop'' denotes the test set constructed from online_shopping_10_cats, including 9,000 comments; and ''Book'' denotes the test set with 3,000 comments about book reviews.   Tables 13 and 14, the proposed model generalizes well when oriented to comments from other platforms. The model achieves scores above 90% on both the Weibo comments and online shopping comments, and achieves the best performance on the WB12K test set (an accuracy of 95.25% accuracy and an F1 score of 95.22%). These results demonstrate that the GNN-LSTM model has good sentiment classification for short comments on Weibo and can also optimally perform for other open comment areas.

As shown in
In addition, 12 sets of data were constructed on dataset 1 with a data volume of 10,000 to 120,000 in increments of 10,000. In addition, 10% of the comments were selected from each of the datasets for testing. The test results of relevant models on 12 groups of datasets with different sizes are compared, as shown in Fig. 8, to analyze the effect of data volume on model performance. Fig. 8 shows the accuracy curves and F1-score curves of benchmark methods on 12 groups of datasets. GNN-LSTM achieves the optimal and smoothest curve in the comparison experiment, which shows that the method of constructing semantic graphs based on internal connectivity and designing a spatial graph filter based on it has a more stable performance on datasets of different sizes. GNN-LSTM still has an effective classification effect on small sample datasets.
As shown in Tables 13 and 14, the GNN-LSTM model outperforms the other models except for the Book dataset. Possible reasons for this result are as follows: GNN-LSTM constructs semantic graphs based on the dependent syntax of each short text for graph convolution and relies on internal connections of texts without establishing external connections between texts, which may affect performance. However, compared with the graph construction method of embedding words and documents throughout the corpus, the composition of GNN-LSTM is more flexible, saves a large amount of storage space, and reduces spatial complexity. Therefore, GNN-LSTM achieves better stability and generalizability.

H. ANALYSIS OF ADVANTAGES AND LIMITATIONS
Compared with benchmark methods, the proposed method has the following advantages: 1) Compared with methods in [38] and [39], an interpretable feature space is constructed, which reduces the dependence on sequences and extracts both semantic and syntactic features of sentences.
2) Compared with methods in [29] and [40], the composition method based on the entire corpus has been abandoned, and a more flexible composition method of single text is used instead to save storage space. The use of a spatial graph filter also reduces the computation.
3) Compared with the method in [33], the proposed method does not use the semantic graph as external knowledge for feature supplementation but rather completes the feature extraction directly on the semantic graph. The constructed graph filter and feature space have a better interpretation, which completes the intuitive transformation from text classification to graph classification.
However, there are several implementation challenges with the proposed method.
1) When constructing semantic graphs, only certain special characters are filtered in the preprocessing stage to perform complete dependency parsing, but stop words cannot be removed. Although complete structural features are retained, a lot of noise exists in the graph data, which may affect model performance.
2) The GNN algorithm implemented by the MPNN framework requires considering both feature passing and state updating in the optimization stage, which increases the difficulty of model optimization. 3) The method of single-text composition only describes the topological structure inside the text but fails to establish the connection between sentences and obtain the overall features of the corpus. With uneven data quality, this issue is more likely to affect model accuracy.
These three points highlight the limitations of the proposed method compared with the other methods. In future research, we plan to use optimization strategies to address these limitations.

V. CONCLUSION & FUTURE DIRECTION
Currently, the number of user comments on social platforms such as Weibo are rapidly increasing and contain spoken expressions. In this study, we develop and test a sentiment analysis model based on GNN-LSTM. While extracting semantic features, the syntactic features are retained to enhance the interpretability of the feature space. GNN-LSTM captures continuous features and optimizes the sentiment classification performance. The LSTM model filters the graph noise, and effective aggregated information is retained. Experimental results show that the GNN-LSTM model achieves good generalizability for the open comment area of the Internet and can effectively accomplish sentiment analysis with short texts.
In future work, we plan to use two strategies to optimize the proposed model's performance to address the learning limitations on heterogeneous semantic graphs due to the strong homogeneity assumption of the GNN model. These strategies are 1) constructing relationships between texts enriches the feature space, and 2) introducing higher-order adjacent features enhances the features of semantic graphs and increases the perception range of convolution.