A Retrieval-based Chatbot for Customer Service Using Large Language Model

The surge in service requests at the Information Technology Service Center at the beginning of each semester poses a substantial strain on human resources, necessitating innovative solutions for efficient service management. With advancements in Natural Language Processing (NLP), AI chatbots have been introduced to address this challenge. This paper implements an AI chatbot utilizing OpenAI GPT-3.5 on the Information Technology Service Center website, discovering its capabilities of handling miscellaneous tasks. Our key contribution is to enable information retrieval from a question-answering database and provide a human-like bot service. Given a specialized prompt, the chatbot undergoes information-retrieving and answer-generating processes to achieve its goal. We use accuracy and BERT scores to evaluate the quality of the answers, and the results of 0.8844 in accuracy and 0.8517 in BERT F1-score demonstrate the chatbot’s capabilities in handling service requests as a replacement for human resources. Lastly, this paper highlights possibilities for further extensions that can be made to enhance the chatbot’s reliability and improve the overall user experience.