Evaluating the Performance and Accuracy of Supervised Learning Models on Sentiment Analysis of E-Wallet

This study has purpose to evaluate the performance and accuracy of supervised learning Models used on Sentiment Analysis of E-Wallet. User comment data was taken using the web scraping method on 4 major E-wallet applications in Indonesia namely, Ovo, Dana, Doku, and LinkAja. The data was taken in January-May 2023 with the amount of data after preprocessing are 11267 with 6349 negative labels and 4918 positive labels. Data labeling uses a star scale that has been pinned by the user, 1-3 stars are labelled negative and 4-5 stars are labelled positive labels. The labeling results were tested using supervised learning model including SVM (Support Vector Machine), Multinomial Naive Bayes, Bagging with Multinomial Naive Bayes, and Random Forest algorithms. The performance of these algorithms is measured using Precision, Recall, F1-Score, and Support. The accuracy of the algorithms is also evaluated using train accuracy score, test accuracy score, train ROC-AUC Score, test ROC-AUC score, area under precision-recall curve, and area under ROC-AUC. This study shows that labeling generates a significant value, which means that the user’s negative and positive comments need to be considered by the E-wallet manager in order to improve the quality of the system and services.