Combinatorial Test Design Model Creation using Large Language Models

In this paper, we report on our initial experience in using Large Language Models (LLMs), which continue to impact a growing multitude of domains, further expanding machine learning applicability. One possible use case is to apply LLMs as a tool to help drive more optimized test coverage via assisting to generate Combinatorial Test Design (CTD) models. This can lower the entry barrier for new CTD practitioners by requiring less subject matter expertise to generate a basic CTD model. In this paper we report on our initial experience in using LLMs to generate a base CTD model and analyze the usefulness of the approach. In common testing scenarios, the LLMs easily provide the necessary attributes and values that are needed to define the CTD model. Prompting the LLM for additional use cases is useful in highlighting possible interactions and determining constraints of the attributes identified in the first stage. Combining the two stages together facilitates the creation of base CTD models.