Public Sentiment Analysis in Twitter Data for Prediction of a Company's Stock Price Movements

There has recently been some effort to mine social media for public sentiment analysis. Studies have suggested that public emotions shown through Tweeter may well be correlated with the Dow Jones Industrial Average. However, can public sentiment be analyzed to predict the movements of the stock price of a particular company? If so, is it possible for the stock price of one company to be more predictable than that of another company? Is there a particular kind of companies whose stock price are more predictable based on analyzing public sentiments as reflected in Twitter data? In this article, we propose a method to mine Twitter data for answers to these questions. Specifically, we propose to use a data mining algorithm to determine if the price of a selection of 30 companies listed in NASDAQ and the New York Stock Exchange can actually be predicted by the given 15 million records of tweets (i.e., Twitter messages). We do so by extracting ambiguous textual tweet data through NLP techniques to define public sentiment, then make use of a data mining technique to discover patterns between public sentiment and real stock price movements. With the proposed algorithm, we manage to discover that it is possible for the stock closing price of some companies to be predicted with an average accuracy as high as 76.12%. In this paper, we describe the data mining algorithm that we use and discuss the key findings in relation to the questions posed.