The Impact of Text Preprocessing on Personality Analysis on Twitter Social Media Posts in Bahasa Indonesia

Many studies have addressed preprocessing, however none have explored the impact of various preprocessing techniques or determined the optimal combination for personality analysis. Therefore, this research focuses on implementing various preprocessing techniques to identify which combination yields the best prformance in personality analysis. The preprocessing process involves normalization, cleaning, stopword removal, and stemming. Additionally, this study incorporates feature extraction using N-Gram features, including unigram, bigram, and trigram. Based on the results of several testing scenarios, the best system performance was achieved using the combination of preprocessing techniques: case folding + normalization + cleaning + stemming, resulting in an accuracy of 79%. Furthermore testing with the application of unigram feature extraction yielded better accuracy compared to bigram and trigram. Normalization plays a significant role in preprocessing, as words on Twitter often use non-standard language, and the purpose of normalization is to transform non-standard words into standard ones.