Development of sentiment indicators using both unlabeled and labeled posts

Extracting useful information for market trend analysis automatically from textual data is an important issue in the financial field. Textual data from Yahoo finance bulletin board posts is one of the valuable resources. From these posts, we can obtain the useful information such as investors' sentiments and discussions between investors. On these boards, the number of untagged posts is more than ten times that of posts with sentiment tags. Thus, we believe that sentiment indicators extracted from both unlabeled and labeled posts can represent investors' sentiments better than those extracted from only labeled posts. However, no other previous work had focused on using unlabeled posts. In this paper, we propose a framework for extracting sentiment indicators from both labeled and unlabeled posts on stock message boards. First, we develop models for predicting sentiment tags on posts using tagged posts as training data. Then, we assign polarity scores to unlabeled posts by using the prediction models and develop sentiment indicators using the polarity scores of all the stock message board posts. Our experimental results show that the proposed sentiment indicators are more strongly correlated with stock prices than those developed using only the labeled posts. We also develop a novel indicator called GAP using the proposed sentiment indicator and demonstrate experimentally that GAP is helpful for evaluating the reliability of the sentiment tags for stock message board posts.