Combinatorial Interaction Testing for Test Selection in Grammar-Based Testing

Systematically enumerating derivations of a grammar yields for realistic grammars test sets that are to large to be tested with reasonable costs. Existing reduction techniques for grammar-based testing guide the enumeration process to restrict the number of generated test cases. However, they do not provide a rule coverage criterion, i.e., they do not aim at providing a test set that ensures coverage of t-wise rule combinations. Selecting derivations such that all possible t-wise rule combinations are covered with at least one test case allows for the generation of small test sets. We employ combinatorial interaction testing to generate the test sets and derive the necessary test specification from the grammar specification automatically. Our evaluation results are twofold. First, they demonstrate the efficiency of our approach. Second, they reveal shortcomings of existing tools for combinatorial interaction testing with constraints.