Exploring the Use of LLMs to Reduce the Discarding of MBT Test Cases

Model-Based Testing (MBT) enables the automated generation of test suites from requirement models. However, the frequent changes in agile development often lead teams to indiscriminately discard existing test cases, undermining the efficiency of Test Case Maintenance (TCM). This practice results in the loss of valuable test artifacts and escalates costs due to redundant test generation. Previous research has explored test case reuse through distance functions, but this strategy often suffers from low precision and misclassification. These issues lead to an excessive number of test cases being incorrectly considered reusable. In this paper, we investigate the use of Large Language Models (LLMs) to improve test case management. Through an empirical study on two industrial systems, we analyzed the performance of 13 well-known LLMs in classifying the impact of use case edits using CoT (Chain of Thought)/ToT (Tree of Thought) prompting and Naive-RAG strategies. Our findings indicate that seven of these models effectively reduced the unnecessary discarding of test cases by accurately identifying high-impact requirement changes, achieving a 7% improvement over distance functions. This resulted in a more precise, reliable, and efficient TCM solution within MBT. However, compared to distance-function-based strategies, LLMs exhibited slightly lower recall, performing 6% worse in test case reuse and reduction information loss.