Open Benchmark Testbed to Evaluate Effectiveness of Path-based Tests

In path-based testing, various test coverage criteria can be used to generate test paths from a system model. The question of the realistic economic effectiveness of these individual criteria deserves further investigation, as the answer strongly depends on the presence of defects in a system, topology of the model of a system under test, and other factors. This study presents an open benchmark testbed for measuring the effectiveness of test paths in detecting artificially introduced defects in a tested system. This framework offers a good level of scalability for various experiments in this field. To document its functionality and added value, an example use case comparing the effectiveness of test paths satisfying Edge, Edge-pair, Test Depth Level 3 and Prime Path coverage for the detection of 75 artificial defects is presented.