A Comparative Study of RAG and Fine-Tuned Transformer Models for Domain-Specific Chatbots

This paper explores the development of a customized chatbot for interactive website support, implemented using two separate methodologies: a Retrieval-Augmented Generation (RAG) framework and a fine-tuned transformer-based model. The chatbot is designed to provide accurate, domain-specific responses for the IRIS Club at MIT-WPU, handling questions about club activities, projects, events, and member information. We perform a comparative analysis of the two methods based on factors such as implementation difficulty, response accuracy, and resource consumption. The RAG-based solution integrates LangChain, HuggingFace embeddings, and the LLaMA-3 model accessed via Groq’s API, whereas the alternative leverages a DistilBERT model trained on relevant data. To evaluate performance, we use metrics like cosine similarity, BERT score, ROUGE scores, and Flesch-Kincaid readability. Our findings indicate that both strategies are viable, each offering distinct advantages and limitations in terms of precision, ease of development, and computational efficiency.