Clue Propagation Based on Non-Adjective Opinion Words for Handling Disconnected Propagation in Product Reviews

Recently, much research has focused on explicit aspect extraction. User reviews in the textual form are unstructured data, creating a very high complexity when processed for sentiment analysis. Previous propagation approaches proposed in this area mainly focused only on adjective-and-noun-based relations. By using only a small seed of opinion words, there is a possibility that the propagation can be disconnected, resulting in several aspects and opinions remaining unextracted. To overcome the aforementioned problem, we propose clue propagation method by using several clues to successfully identify aspects and opinions to extract explicit aspects, including noun-and-verb-based relations. The utilization of more than one clue in the propagation prevented premature termination, extending the scope of the extraction. We evaluated our proposed method by comparing it with several state-of-the-art methods. Empirically, the experimental results showed that the added more clues in clue propagation method could extract aspects and opinions that were not extracted in previous studies. Clue propagation can overcome the possibility of disconnected properties in the propagation method.


I. INTRODUCTION
In opinion mining, aspect extraction plays an important role in extracting aspects and sentiments from opinionated text. Aspect extraction is quite challenging because aspects and sentiments can be expressed either explicitly or implicitly. Most researchers have focused on extracting explicit aspect expressions wherein aspects and sentiments are expressed explicitly [1]- [5]. Moreover, the current work primarily focuses on extracting nouns and adjectives as aspect-and-sentiment pairs [1], [4]- [6]. However, verb, noun, and adverb expressions can also imply opinions [2], [7]. For example, I enjoy shooting with this camera, in which case there is an opinion that implicitly refers to the camera expressed by the verb enjoy. If we can successfully extract aspects, we can overcome the most significant obstacle in sentiment analysis.
The associate editor coordinating the review of this manuscript and approving it for publication was Mu-Yen Chen .
Typically, explicit aspect expressions can be extracted using frequency-based [2], [6], and lexicon-based approaches [7]. The frequency-based approach considers frequent nouns as potential aspects, and the lexicon-based approach utilizes the opinion lexicon to extract aspects and their sentiments. Both methods are relatively simple and effective [2]. However, the frequency-based approach cannot identify aspects that are not frequently mentioned in reviews. There is a high dependency on a large lexicon opinion to extract potential aspects in the lexicon-based approach. To overcome this drawback, Qiu et al. developed a Double Propagation (DP) method that applies a bootstrapping and propagation method to effectively extract explicit aspects by using only a small seed set of opinion words [1]. The DP method utilizes dependency relations to extract explicit aspect expressions simultaneously. It consists of four stages of propagation [1]: (1) extracting aspects using sentiment words, (2) extracting aspects using extracted aspects, (3) extracting sentiment words using extracted aspects, and (4) extracting sentiment words using both given and extracted sentiment words. VOLUME 10, 2022 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ Note the following opinionated texts as an illustration: (1) Canon gives a great picture; (2) the picture is amazing; (3) it has more storage to store high-quality pictures and recorded movies; (4) the software is amazing; (5) this camera is a winner; and (6) I recommend this camera. In the DP method, a small number of seed words are used to extract aspect expressions, consisting of several adjectives representing positive and negative opinions. For instance, consider adjective seed words {good, great, bad, worst}. By propagating the adjective opinion seed great, the picture in sentence (1) can be extracted. The opinion word amazing in sentence (2) can be extracted by using the extracted aspect picture. The movie in sentence (3) can be extracted by using conjunction that relates the extracted aspect picture and movie. Furthermore, the software in sentence (4) can be extracted by using the extracted opinion word amazing. However, the camera in sentence (5) never relates to the extracted aspects and opinions, so it cannot be extracted. Similarly, in sentence (6), the camera is not successfully extracted, although its opinion is implied in the verb recommend. Even when using the above propagation steps and established dependency rules, several aspects remain unextracted.
Although effective, the illustration above shows the drawback of aspect extraction methods by simply using adjective opinion seed words. Due to the disconnected property of the propagation method, several aspects cannot be extracted. Therefore, it is not sufficient to rely solely on nouns and adjectives as potential aspects and opinions. Previous research has shown that opinions are not always expressed in adjectives [8]- [11], and aspects are not always expressed in nouns or noun phrases [2]. Poria et al. conducted Implicit Aspect Clues (IACs) that were used to identify aspects indirectly [12].
We observed that other clues could be used to identify aspects and opinions. Therefore, we propose a clue propagation algorithm to solve disconnected propagation in the previous work by introducing more clues that point toward the potential aspects and opinions. The proposed approach uses a small set of seed words, as in the research conducted by Qiu et al. [1], to be used as ''clues'' to the opinion words.
The main contribution of this paper is that we added three significant potentials ''clues'' in addition to ''clue'' of adjective opinion words in order to address the aforementioned problem. The clues are as follows: (a) verb expression clues to encompass all opinions expressed in non-adjective words; (b) entity expression clues to encompass all aspect and opinion candidates that cannot be propagated from the extracted aspects and opinions; and (c) adverb clues. The initial stage of the proposed method is to extract all aspect and opinion candidates based on these clues and then propagate and perform filtering and validation to eliminate noise terms. These clues and their associated propagation rules can extend the coverage of extracted aspects and opinions. Additionally, it can overcome the disconnected propagation observed in previous studies. We compared our approach to both the DP and state-of-the-art methods. In Section V, we show that adding these clues to clue propagation method results in significant performance improvement.
The remainder of this paper is organized as follows. Section 2 presents previous works related to rule-based methods for aspect extraction. In Section 3, the clue propagation is described in detail. Sections 4 and 5 describe the experiments and the discussion of the results, followed by the limitations and conclusions in Sections 6 and 7.

II. RELATED WORKS
Many methods for extracting aspect expressions have been developed in previous studies. Rule-based methods utilize the grammatical relationships between aspects and their corresponding opinions [1], [12]. This method is effective in generating sufficiently high recall; thus, it has been widely developed in numerous research studies [1], [12], [13]. The bootstrapping technique is one of the most widely used in rule-based methods [1], [12]- [16]. Qiu et al. proposed a bootstrapping method by exploiting dependency relations and then synthesized the method into the DP method to extract explicit aspects simultaneously [1]. The DP method requires only a small set of adjectives to extract aspects through four stages of propagation. The propagation uses several rules of dependency relations and has been proven to effectively extract explicit aspect expressions [1]. The DP method has a major advantage in that it only requires an initial opinion lexicon to start bootstrapping. However, since this method uses adjectives as seed opinion words and nouns as the target aspect, the extraction only produces explicit aspect expressions in noun/noun phrases.
Meanwhile until recently, existing research has mainly focused on explicit aspect extraction indicated by opinion adjectives [1], [6], [17]- [19]. However, this is not always the case, as opinions can be expressed as adjectives and verbs. Several studies have stated that nouns and verbs could also imply opinions [2], [7], [20]. Noun-based opinion identification was first developed by Riloff et al. [21], who used bootstrapping algorithms that exploit extraction patterns to learn sets of subjective nouns, while Zhang et al. [7] focused on noun and noun phrase identification, which implies opinions by using an opinion lexicon. In addition to noun-based aspect extraction, some studies have developed a verb-based approach to extract aspects [2], [8], [10], [22]. Other studies have also stated that a verb can implicitly represent opinions [8], [23]- [25]. However, there are numerous challenges in deriving aspects and their associated opinion words based on nouns and verbs [2], [8], [10], [22].
To address the limitation of the previous works, we adopted a propagation method based on dependency relations rules, namely the ''clue propagation method,'' to overcome the aforementioned drawbacks [1], [12], [14].

III. THE PROPOSED METHOD
Intuitively, opinions are expressed as adjectives. Thus, aspect extraction can produce a high recall simply by using a small set of opinion seed words [1]. However, using adjective seed words as a ''clue'' in aspect extraction cannot identify all aspects because opinions are not always expressed as adjectives. As in the following example: this phone is a winner, and the opinion is contained in a noun winner. In addition, opinions can be implied by verb expressions. For example, I love this phone, the opinion is implicitly implied in the verb expression love. Similarly, aspects are not always expressed in noun/noun phrases; for example, this phone feels great and looks great too. In this sentence, aspects are expressed in verb expressions: feels and looks. Thus, other clues can be used to indicate aspects and opinions that are undetected by propagating adjective opinion words. Additionally, as explained in Section I, there is an issue with propagation being disconnected because it only depend solely on small seed opinion words.
In response to the drawbacks of the previous propagation methods, we developed a new extraction method by adding more clues that broaden the scope of identified aspects and opinions. Our hypothesis is that adding more clues such as verbs, entity clues and adverb clues can reconnect the disconnected propagation so that more aspects and opinions can be extracted.

A. MAIN IDEA AND STEPS
The main idea of the proposed clue propagation algorithm is that there are always clues that indicate an aspect and/or opinion, in addition to clues such as adjectives. In general, the main steps of the aspect extraction presented in this study are as follows: 1) Identification of aspects and opinion candidates. This step aims to identify all aspects and opinion candidates obtained from the clue propagation that leads to both. The clues consist of adjective opinion words, verb expressions, entity-expressions, and adverb clues that indicate the potential aspect and opinion, resulting in a more thorough extraction. 2) Propagation. It propagates the clues, and extracted aspects & opinions to extract new aspects and opinion candidates. 3) Filtering and validation. Not all aspect candidates generated in the first and second steps are relevant. Filtering and validation are necessary to eliminate noise terms.

B. IDENTIFICATION OF ASPECT AND OPINION CANDIDATES
Four types of clues extract candidates of aspects and opinions. The first type of clue was adapted from previous research [1], whereas the other three types of clues were added as contributions to this research.
1) Opinion word clues. The method begins with a small seed of adjectives representing a positive or negative opinion, as employed in Qiu et al. [1]. We used Hu and Liu's seed opinion lexicon, which contains 654 positive and 1098 negative opinion words [6]. The point of determining opinion words clues is to select adjectives that clearly represent positive and negative opinions. For example, we can use the clue great as a seed for positive opinion and the clue worst as a seed for a negative opinion. 2) Verb-expression clues. In addition to adjective opinion words, we proposed verb expression clues because opinions can also be expressed indirectly in verb expressions, such as {love, like, dislike, hate, recom-mend}. Opinions expressed as verbs generally lead to an entity, object, or a specific aspect. A verb-expression clue is used to cover non-adjectives. This study uses a list of verb lexicons from Wiebe et al. [11], containing 8,222 words, including 1,325 verbs. This lexicon verb can assist in selecting verb-expression clues, as the list contains both positive and negative verbs. 3) Entity expression clues. It is logical to have a ''clue'' of entity expression and/or aspect expression in a collection of reviews that discuss a particular entity. For example, in reviews of a camera product, the word camera is frequently mentioned and several aspect term expressions. We propose entity expression clues that consist of a seed of entity categories, entity expressions, aspect categories, and aspect expressions. Our primary aim is to obtain a more comprehensive extraction coverage to reduce the number of aspects that cannot be propagated. The entity expression clues used in this study were manually collected from the dataset. 4) Adverb clues. Additionally, adverb clues can also be used to identify opinions, such as: {absolutely, really, highly, extremely, well}. In this paper, negation modifiers that can reverse the polarity of related words, such as: {never, nothing, not, no} were also used in clue propagation. This study utilized a list of 330 adverbs, including 132 positive and 198 negative adverbs [11].

C. CLUE PROPAGATION
This section describes the clue propagation method, which consists of two subsections: the propagation process and propagation rule.

1) PROPAGATION
It propagates clues, extracted aspects, and opinions to extract new aspects and opinions as in the DP method. In addition to using opinion words, verbs, and adverb clues, we also utilize entity clues, including entity expressions and aspect expressions. Clue propagation employs a set of rules based on dependency relations. Unlike Qiu et al. [1], who used Minipar as the sentence parser, this study utilized the Stanford parser to extract the dependency relations. As an illustration of the clue propagation method, consider the following sentences: 1) Canon gives great picture.
2) The picture is amazing.  3) It has more storage to store high quality picture and recorded movie. 4) I love Canon and recommend this camera.

5) Canon is a masterpiece.
Suppose we have {great, bad} as a seed of adjective opinion words so that we can extract the aspect picture in sentence (1). The extracted aspect picture is propagated to extract opinions amazing and high quality in sentences (2) and (3) respectively, as well as extract aspect movie in sentence (3). The extracted aspect movie can lead to the opinion word recorded.
Suppose also we have recommend as a verb expression clue and Canon as a pre-defined entity clue. Thus, the clue propagation can identify camera as an aspect in sentence (4) and extract masterpiece as an opinion word in sentence (5).

2) RULES OF CLUE PROPAGATION
We define four groups of rules based on the types of clues and dependency relations. Let A be a set of extracted aspects and entity clues. Let O be a set of extracted opinions and a set of seed words consisting of opinion word clues, verb expression clues, and adverb clues. Let SDepRel be a set of dependency relations based on Stanford Parser, which represents the dependency relation of the word O (or A).  Table 1.
The following are the four task groups and their propagation rules. 1) Extracting aspect by using opinion word clues, verb expression clues, adverb clues, and/or extracted opinions.
In this research, we modify two rules (R1 1 and R1 2 ) in the existing method (DP) [1] due to the addition of clues used in the clue propagation method. Moreover, we add two new rules (R1 3 and R1 4 ) to expand the scope of extraction and prevent the interrupted propagation. a) Opinion directly depends on aspect (R1 1 ) This rule indicates that opinions directly depend on aspects without any additional words in their dependency path. The DP method only determines adjectives and nouns as targets for opinions and aspects, respectively [1]. However, this paper also incorporates non-adjectives and non-nouns word clues. Therefore we add several constraints to the existing rules in DP (R1 1 and R1 2 )  Because best ∈ O and it depends on Ipod through player with {DepRel ∈ SDepRel} (i.e., nsubj(player,Ipod) and amod(player,best)), then Ipod can be extracted as an aspect. c) Aspect directly depends on opinion (R1 3 ) As in Rule R1 1 , this rule indicates that the aspect directly depends on opinion without any additional words in their dependency path, with respect to some constraints. We observe that in addition to a sentiment that depends on the aspect (Rule R1 1 ), there is also an inverse relation between aspect (A) and sentiment (O 3) Extracting aspects by using entity expression clues or extracted aspects.
a) Aspect directly depends on entity expression clues or the extracted aspects (R3 1 ). This rule uses a conjunction dependency relation to extract another aspect that has a direct dependency. Rule R3 1 : Given a set of aspects A 1 , DepRel ∈ {NN }, and DepRel ∈ {conj}, then we can obtain A 2 as the extracted aspect. Example 9: The camera and the picture are cracked.
We can obtain picture as the extracted aspect based on an entity aspect camera. b) Aspect indirectly depends on entity expression clues or the extracted aspect (R3 2 ). Rule R3 2 : Given a set of aspects A 1 , DepRel (X,A 1 ) and DepRel(X,A 2 ), such that POS(A 1 ) ∈ {NN }, POS(A 2 ) ∈ {NN }, and DepRel ∈ {SDepRel}, then we can obtain A 2 as the extracted aspect. Example 10: Canon has a great lens. VOLUME 10, 2022 Aspect Canon indirectly depends on the aspect lens through the word has. Therefore, if we have an entity aspect Canon, we can obtain the lens as an extracted aspect. For a given opinion word portable, the extracted opinion word is small because it has an indirect amod dependency relation with a portable through word camera (i.e., amod(camera, small) and amod(camera, portable)).
The detailed algorithm for the clue propagation is shown in Algorithm 1. The clue propagation method requires four types of clues as the input: entity clues {E}, opinion word clues {Op}, verb expression clues {V}, and adverb clues {M}. There are two main steps: extraction of aspect opinion candidates and the propagation process. The first step uses clues to extract a list of aspect candidates {A} and opinion candidates {O}. Then, the propagation phase extracts all of the related aspects and opinions by using the list of {A} and {O}.

D. FILTERING AND VALIDATION
In general, aspect extraction generates noise terms in addition to the extracted aspects and opinions. Because the clue propagation method utilizes several clues, the number of candidate aspects and opinions becomes more numerous. Although the clue propagation method can minimize this by utilizing specific rules of dependency relations, filtering and validation are still needed to separate the noise from the desired information. We employ a three-phase pruning algorithm to identify aspect and sentiment word noises, which are also extracted using the clue propagation method. The first phase involves the identification of aspects and non-aspects using the frequency-based pruning method. The second phase identifies the phrase and non-phrase aspects. The last phase is the sentiment word validation. The three-phase pruning algorithm is presented in Algorithm 2. Each phase is executed in the following order: 1) Identification of aspects and non-aspects The frequency-based pruning method is applied by ranking the aspect candidates generated using the clue propagation method. It assumes that a target aspect will appear with a frequency greater than the noise aspect so that the aspect candidates whose frequencies are below a threshold can be eliminated. The threshold was set to five to perform this pruning [27].

2) Aspect phrase identification
Aspect candidates produced by the clue propagation method are individual terms, such as battery, weight, and size. However, a target aspect can be a phrase (such as battery life); it is essential to identify it from an individual word. Identifying whether an individual term is a part of a phrase is performed through the notion of a syntactic phrase. It is defined as a word sequence that is covered by a single subtree in a syntactic parse tree. This syntactic phrase is determined by the noun embedded within it and serves as its head. Thus, if the aspect candidate is the head term, it is substituted by a noun phrase.

3) Opinion validation
This phase utilizes the external knowledge sources of Bing Liu's opinion lexicon [28] and Sentiwordnet [29]. Opinion candidates will be matched with a lexical collection from these knowledge sources. It eliminates opinion candidates, including nouns, verbs, and adverbs, which are not opinion words. For example, the opinion word candidate digital in the phrase digital camera is removed because it does not represent an opinion.

IV. EXPERIMENTS AND RESULTS
In this section, we describe the experiment dataset and the results.

A. DATASET
This paper uses various review sentence domains to obtain extraction rules that can cover various variations in review sentences. We used the annotated customer reviews of electronic products that have been widely used for aspect extraction, collected from Amazon.com and Cnet.com [6]. The dataset consists of seven electronic products: four digital cameras, one cellular phone, one DVD player, and one mp3 player. The human tagger manually labeled the sentences to determine the features and polarity of each sentence. The features are most explicit in sentences, for example, the picture in the picture is amazing. The implicit features such as size in It fits in a pocket nicely are also easy to identify by the human tagger. Additionally, we utilized a benchmark dataset from SemEval-2014 for the Laptop domain and a benchmark dataset from SemEval-2016 for the Restaurant domain [30].
The following are the examples of the dataset: 1) camera[-2]## I want to start off by saying that this camera is small for a reason. 2) size [-2], camera[-1]## Some people, in their reviews, complain about its small size and how it doesn't compare with larger cameras.
3) ## I bought this little guy a few weeks back, and I have to say that I never had so much fun with a new toy like this. The beginning of each sentence is marked with ##. The first review is annotated with camera as an aspect with negative polarity and opinion strength of 2. Opinion strength varies between 3 (strongest) and 1 (weakest). Note that the strength is subjective. We did not use the opinion strength in our study. The second review has two aspects: size and camera, both of which have negative polarity. Unlike the first sentence, the third sentence does not contain any aspects. Table 2 shows detailed information on the datasets, including the number of sentences and the number of aspects.

V. RESULT AND DISCUSSIONS
To evaluate the performance of the proposed method, we compared the results of the clue propagation method with the DP method as a baseline [1]. It is more critical to analyse the performance of relevant elements rather than the entire dataset. Therefore, we evaluated the results using microaveraging precision and recall. The microaveraging strategy performs the sum of the terms of the evaluation measures. Precision indicates the proportion of data points that the model indicates are relevant that are actually relevant, whereas recall indicates the model's ability to locate all relevant data in the dataset. These evaluation metrics are more appropriate for relevant data than for irrelevant ones. Furthermore, these metrics have also become the de-facto standard for aspect extraction.
In this context, true positive (TP) is the number of aspects that are extracted correctly, false positive (FP) is the number of aspects that are extracted, but these are not aspects, and false negative (FN) is the number of aspects that are not successfully extracted. Aspect extraction is naturally a problem with imbalanced data in which the number of extracted aspect terms is way much smaller than that of non-aspect terms. F-measure is the most appropriate performance measure for this problem and has become the standard for measuring performance in information extraction problems. Because the original corpus and source code were not available, the DP method was reimplemented for comparison. The performance comparison results are presented in Table 3. Table 3 shows that our approach outperforms the baseline (DP method) in all datasets [1]. The proposed method improves the performance by 3%-10% compared to the  baseline using paired t-test. It has been observed that the p-value in paired t-test is .02, which is less than .05. It indicates that the improvement of our proposed method is statistically significant at the confidence level of 95%. The performance of the proposed method improves because it uses more clues to extract additional potential aspects and opinions, which are not propagated using the baseline method. Furthermore, the extraction rules also affect the performance, which has a broader scope because they allow noun and verb-based relationships. Thus, the extraction process can identify and extract aspects and opinion candidates that were not extracted using the baseline.
On the benchmark Laptop and Restaurant datasets, as shown in Table 4, our proposed method (Clue Propagation) still has improved over the baseline (DP) method by approximately 7.4%. While it is competitive with the stateof-the-art of RN-CRF method (i.e., our method is better on the Restaurant dataset but not for the Laptop dataset), the Clue Propagation is not better than the rest of state-of-theart deep learning methods. Nevertheless, in some cases, Clue Propagation offers several benefits over the currently popular deep learning methods. First, as a rule-based approach, the Clue Propagation is more practical in many real-world situations. Because the rules are hand-crafted and inferred from broader knowledge about the relationship between an aspect and its opinion, they can be practically applied to many domains directly with much less effort for adaptation if required. It also enables the Clue Propagation to be less affected by an unbalanced dataset compared to the machine learning-based approach. Second, although developing the rules requires more effort, the rule-based method does not require training data and so does not incur the high computational and time costs for performing the training process, such as in deep learning. Utilizing rule-based and deep learning methods entails trade-offs in terms of efficiency, effectiveness, and model cost. The rule-based method is preferred if practicality to get the job done with moderate but still tolerable accuracy faster is more important than merely obtaining the best model accuracy.

VI. LIMITATIONS
Theoretically, by using more clues, the proposed method should extract all aspects and opinions contained in the reviews compared to the baseline method. However, our proposed method has several drawbacks. Although the experiments showed a performance improvement, some aspects could not be extracted. This may be caused by two factors: the type of expression and the sentence structure.

A. TYPE OF EXPRESSION
The rule-based approach has the limitation of not extracting sentences that do not contain expressions of explicit aspects and sentiments. As mentioned in the literature, apart from    [40]. For example, it can be seen in the review sentence: It doesn't include a memory card, which expresses the wishes of reviewers about aspects of the memory card or states that the product lacks a memory card.
Another example can be seen in the sentence: after considering the marketplace's needs and further research, I decided on the s100. The expression of the reviewer's sentiment is implicitly conveyed in the overall meaning of the sentence above. Likewise, the review sentence: I can only hear the sound but no picture!, it does not contain expressions of sentiment explicitly in the picture aspect.

B. SENTENCE STRUCTURES
The structure of a review strongly influences the rule-based approach. Some sentence structures cannot be handled using this approach, as follows: 1) Incomplete sentences.
Some reviews are sometimes stated as incomplete sentences. For example, very bad quality; first off, the battery; no GPS. These reviews do not explicitly mention aspects or opinions; therefore, the proposed method cannot extract either aspects or opinions. 2) Compound sentences.
It is relatively difficult to extract a compound sentence using the syntactic rule-based method because opinions are implied in a sentence's overall meaning. Although the clues used in this paper are quite comprehensive, clue propagation could not relate the clues to the potential aspects and opinions in compound sentences. As shown in Figure 1, the syntactic-based approach cannot extract the relation between the clue embarrassing, the aspect DVD player, and the term no longer working. Similarly, as shown in Figure 2 and Figure 3, the relevant aspect and its opinion cannot be extracted because the opinion is implied in the overall sentence. Figure 2  indicates the lack of relation between the aspect DVD player and the positive opinion in perfect condition, while Figure 3 shows that the complex sentence structure in dataset DVD Player prevents the opinion disappointed in the aspect DVD to be extracted. A compound and complex sentence such as the examples above can be easily found in the DVD player and MP3 Player reviews, causing the precision of the results to be relatively lower.

3) Complex noun phrase
A complex noun phrase frequently contains a mandatory head in addition to a determiner, pre-modifier, and post-modifier. A pre-modifier precedes the head of a sentence. Pre-modifiers are frequently adjectives; however, other nouns can also modify the head, in which case the premodifying noun may be followed by a premodifying adjective. Post-modifiers include relative clauses, non-finite clauses, prepositional phrases, adverbs, adjectives, and noun phrases in apposition. For example, a complex noun phrase orecchiette with sausage and chicken in the Restaurant dataset: our agreed favorite is the orecchiette with sausage and chicken (usually the waiters are kind enough to split the dish in half, so you get to sample both meats). Another example is the aspect ''selection of meats and seafood'' in the sentence ''don't go alone-even two people isn't enough for the whole experience, with pickles and a selection of meats and seafood,'' as shown in Figure 4. 4) Noise terms.
There are several candidate aspects paired with adjectives that are not opinions. For example, I found the integrated digital camera to be very nice. The aspect camera has a fairly high frequency of occurrence and will not be ignored. However, the camera is paired with the adjective digital, which is not an opinion word but is extracted by the clue propagation method. This word is considered to be noise and degrades the performance. Thus, the post-processing task is the key to determining the final performance of the aspect extraction. The more detail that can be separated from the relevant and irrelevant aspects, the higher the performance. In addition, the proposed methods do not cover semantic aspect extraction. Thus, the aspect phone and opinion small in the following sentence are also extracted and will not be ignored: no more hassles using the small phone keypad.

VII. CONCLUSION
We experimentally show that our proposed method can effectively overcome the issue of disconnected propagation by using only a small seed of opinion words. Since the clue prop-agation method uses verb clues as a seed of clues, implicit opinion expressions represented as verbs can be extracted. Additionally, this improvement broadens the scope of aspect and sentiment extraction, so it is not limited to explicit aspect extraction indicated by opinion adjectives. However, there remains ample room for improvement, as our approach struggles where the semantics is still ambiguous or the domain of the dataset is limited. Despite the experiments that the rule-based method does not outperform the most recent deep learning methods, it is competitive in real-world scenarios in which we work in various domains. It is also preferred if we want a practical solution with reasonable accuracy.
In future work, we plan to expand the scope of the extraction to opinions that are implicitly contained in product reviews, including expanding the experimental data set as well as handling noise terms.