Enhancing Query Classification in Chatbot Using LSTM

This study presents the design, implementation, and evaluation of an LSTM-based model for classifying user queries in a cleaning service chatbot. The dataset comprised 50,000 queries, 10% of which were removed as irrelevant or duplicate, leaving 45,000 cleaned queries for training (80%) and testing (20%). Text preprocessing steps included lowercasing, stop-word removal, lemmatization, tokenization, padding, and word embedding using FastText. The model architecture featured a Bidirectional LSTM with 256 neurons, an attention layer for identifying key terms, and a Softmax output layer for classification across 10 categories. Trained with categorical cross-entropy loss and the Adam optimizer over 30 epochs, the model achieved high accuracy metrics, including an F1-score of 92.6% and an AUC-ROC of 0.95. Analysis of errors revealed common misclassifications between similar categories, such as "post-renovation cleaning" and "deep cleaning," prompting improvements through expanded training data and data augmentation. The deployed model has enhanced query handling efficiency and service quality, proving its suitability for production use.