Dear LLM, Why Are They Complaining? On Root Cause Analysis (RCA) In Customer Service Automation Using Large Language Models (LLMS)

The goal of Root Cause Analysis (RCA) is to determine the underlying concern from a host of concerns. In customer service, RCA can help organizations identify fundamental issues in customer complaints and dissatisfaction. Traditional RCA methods relied on human analysis, often prone to subjectivity and bias. Moreover, processing a large volume of complaint narratives demands significant resources. With datadriven approaches like machine learning, the efficiency of RCA has improved. However, these approaches require domain expertise and training data for accurate performance. Large Language Models (LLMs) can mitigate this challenge by learning from limited examples through few-shot learning or without any examples through zero-shot learning. This research introduces a dataset for RCA detection collected from realworld customer complaint narratives across diverse issues. Six state-of-the-art LLMs, including OpenAI's o1, GPT-4o, Sonnet, and Gemini, were evaluated on their performance in detecting RCA. Llama performed the best in open-ended RCA (55%) whereas o1 performed the best in multiclass RCA (68%). Our findings highlight the potential of LLMs in automating RCA. The limitations of current approaches, like model interpretability and domain-specific generalization, were discussed along with future research directions to enhance RCA capabilities further using LLMs.