A Study on Word Embeddings in Local LLM-based Chatbot Applications

Large Language Models (LLMs) have revolutionized natural language processing, but their deployment often requires substantial computational resources and constant network con-nectivity. There is a growing need for efficient, locally-runnable LLMs with improved word embeddings for various applications, including chatbots. This study focuses on developing a Large Language Model (LLM) capable of running locally without network connections to generate solutions for given queries using improved word embeddings. In this work, two pre-trained models are used and further trained separately to enhance embedding results. The system incorporates additional features such as document-based chat functionality. Our approach develops a dynamic and adaptive NLP system that can assimilate new data from various sources when connected to a network, ensuring up-to-date word embeddings that capture the latest semantic nuances and language trends. The study compares FastText, GloVe, and Word2Vec models, finding that fastText outperforms the others in text generation within the specified time frame. While fastText may require more time to generate larger content, its efficacy in producing better results makes it the preferred choice for this application.