Visualization of combinatorial models and test plans

Combinatorial test design (CTD) is an effective and widely used test design technique. CTD provides automatic test plan generation, but it requires a manual definition of the test space in the form of a combinatorial model. One challenge for successful application of CTD in practice relates to this manual model definition and maintenance process. Another challenge relates to the comprehension and use of the test plan generated by CTD for prioritization purposes. In this work we introduce the use of visualizations as a means to address these challenges. We apply three different forms of visualization, matrices, graphs, and treemaps, to visualize the relationships between the different elements of the model, and to visualize the strength of each test in the test plan and the relationships between the different tests in terms of combinatorial coverage. We evaluate our visualizations via a user survey with 19 CTD practitioners, as well as via two industrial projects in which our visualization was used and allowed test designers to get vital insight into their models and into the coverage provided through CTD generated test plans.