Intent Classification in Chatbots with Human-AI Collaborated Large-Scale Dataset

Intent detection is a pivotal component in the functionality of chatbots, enabling the accurate interpretation of user queries to deliver precise responses. The development of reliable chatbots requires the training of sophisticated models on extensive datasets that comprehensively represent the diversity of natural language. This paper addresses the challenge of achieving high accuracy in intent detection for chatbots and virtual assistants. We present a method based on a carefully constructed dataset of over 500,000 queries, including ten different types of queries. To maintain the integrity of the dataset, we actively curated the queries generated by the advanced language model Turbo ChatGPT-3.5 to ensure dataset quality. The performance of various machine learning models was evaluated on this dataset, with BERT and RoBERTa emerging as the leading models. These demonstrated their efficacy in processing large datasets and capturing complex user query relationships, to facilitate a more accurate understanding of user intent and ultimately enhance user experiences with chatbot. We also discuss future work, including exploring additional models, feature engineering, and real-world integration.