STraVEns: Sentence Transformer Voting Ensemble for Intent Classification-Based Chatbot Model

Natural Language Processing has experienced significant advancements in recent years, leading to the widespread adoption of Large Language Model-based chatbots. These chatbots are popular due to their ability to engage in context-aware conversations. However, deploying LLM-based chatbots can be resource-intensive, making them less suitable for smaller applications or focused tasks. To address this issue, we propose a robust and flexible approach to intent classification for chatbots using STraVEns (Sentence Transformer Voting Ensemble), which includes both hard voting and soft voting ensembles of sentence transformers. Our proposed method aims to improve accuracy and versatility in intent-based chatbots model. We use five sentence transformer models for this ensemble framework: RoBERTa, DistilRoBERTa, MPNet, MiniLM L6, and MiniLM L12, and evaluated our approach by training and testing using four distinct datasets: ATIS, IDE, Small Talk, and CLINC150 which cover a range of scenarios from general conversation to specific tasks and out-of-scope intent classification. The results demonstrate that the STraVEns approach is a promising solution for intent classification-based chatbot model. Results show that our ensemble models outperformed previous benchmarks, achieving the highest accuracy and F1-scores across all datasets. The soft voting method provided flexibility and robustness, while hard voting ensured stability in specific contexts. Overall, our study suggests that ensemble-based approaches can enhance the performance of intent classification chatbots model, providing a scalable solution for various applications.