A Big Data Experiment to Evaluate the Effectiveness of Traditional Machine Learning Techniques Against LSTM Neural Networks in the Hotels Clients Opinion Mining

Context: Nowadays, client reviews on social networks can be a great source of knowledge extraction for strategic marketing planning. In the tourism area, opinions given by hotel clients in tourism social networks can drive improvements in service. In this context, traditional text mining techniques and new deep learning technologies should be tried out to select the best options for classifying opinions. Objective: Evaluate the performance and quality of the LDA (Latent Dirichlet Allocation), Naives Bayes (NB), Logistic Regression, SVM (Support Vector Machine) and LSTM - Long Short-Term Memory Units algorithms in the task of opinion mining of hotel reviews published on the TripAdvisor hotel booking website. Method: An In Vivo Controlled Experiment (Case Study) to compare the performance of the classifiers by means of accuracy, precision, recall, F1-measure and average training and classification times. Results: The LSTM model presented the best results regarding quality metrics. However, it did not present satisfactory results regarding processing time. Conclusion: The LSTM classifier had clearly superior performance when compared to the other evaluated ones. On the other hand, its average training and classification times greater than the others classifiers considered.