LLM-Based Multi-Agent Framework for Troubleshooting Distributed Systems

Effectively configuring distributed systems, particularly those orchestrated by Kubernetes, remains challenging due to inherent complexity. This paper introduces KubeLLM, an LLM-based multi-agent framework designed to automate the troubleshooting of Kubernetes clusters. KubeLLM aims to diagnose configuration errors, analyze root causes, and apply necessary fixes, thereby reducing manual effort and improving system reliability. Our collaborative agents utilize Linux shell commands, remember interaction history, and access domain-specific knowledge via Retrieval Augmented Generation (RAG). We also introduce KubeLLMBench, a benchmark suite for evaluating LLM agents on Kubernetes troubleshooting tasks. Extensive evaluations conducted on a testbed emulating multi-node deployments demonstrate the feasibility and effectiveness of KubeLLM. We compare various LLMs (including Llama 3.3, OpenAI GPT-4o, Google Gemini 1.5 Flash, and o3-mini) across different agent configurations (single vs. multi-agent, memory vs. no memory, selfevaluation vs. no self-evaluation), analyzing performance in terms of accuracy, execution time, cost, and robustness against task-specific failures. Results show that multi-agent approaches generally improve average accuracy. While faster models risk complete failure on certain tasks, more robust models like o3-mini offer higher reliability. Notably, enabling self-evaluation reduces the likelihood of complete task failure, enhancing robustness at the cost of increased execution time. Specific configurations offer strong tradeoffs between performance, speed, cost, and reliability, highlighting the critical need to balance these factors when deploying LLM agents in modern DevOps workflows.