Enhancement of Chatbot Conversation for Network Troubleshooting by Combining LLM and System Log Analysis

Conventional chatbots have limitations in addressing complex and unpredictable issues such as network trouble, and their effectiveness often depends on users’ IT literacy, hindering efficient problem resolution. To address this challenge, this paper proposes a chatbot that enables swift and accurate troubleshooting by having a LLM (Large Language Model) directly analyze system logs from network devices, thereby bypassing reliance on user literacy. To establish the practical viability of this approach, we integrated key log pre-processing techniques into the system to maximize the LLM’s analysis accuracy and performance, and then experimentally evaluated their effectiveness. The evaluation demonstrated that our chatbot, utilizing the Llama model, achieved a practical performance with an analysis accuracy of $86 \%$. This paper not only presents a new form of troubleshooting that directly integrates system logs with LLM but also empirically shows that for its realization, refining the input to the LLM to only essential information through effective pre-processing is crucial.