Investigating the Combination between Pre-processing Technique and Text Vectorization for Machine Learning Model in Investor Comment Classification

Recently predicting the stock price will increase or decrease has been an exciting topic to research for academia and practitioner. Based on our literature review, one of the significant parameters that affect stock price increase or decrease is a sentiment from the news or social media. In this research, we utilize CS-2K composed of 1853 labelled investor comments and conduct an investigation of the combination of data balancing techniques such as SMOTE and SMOTE Tomek and text vectorization techniques, namely TF-IDF and Doc2Vec on machine three selected machine learning linear support vector machine, artificial neural network and XGBoost, after the experiment, we found that the highest weighted average f1-score is when combining SMOTE + TF-IDF in Linear SVM, that obtain 54% weighted average f1-score. The overall experiment proves that SMOTE and SMOTE Tomek improve the weighted average f1-score up to 13% on TF-IDF. However, on the other hand, SMOTE and SMOTE Tomek + Doc2Vec reduce its score up to 5% on every machine learning model when classifying complex investor comments.