The Impact of Equivalent Mutants

If a mutation is not killed by a test suite, this usuallymeans that the test suite is not adequate. However, itmay also be that the mutant keeps the program’s seman-tics unchanged—and thus cannot be detected by any test.We found such equivalent mutants to be surprisingly com-mon: In an experiment on the JAXEN XPATH query engine,8/20 = 40% of all mutations turned out to be equivalent.Worse, checking the equivalency took us 15 minutes for asingle mutation. Equivalent mutants thus make it impossi-ble to automatically assess test suites by means of mutationtesting. To identify equivalent mutants, we are currently investi-gating the impact of a mutation on the execution: the morea mutation alters the execution, the higher the chance of itbeing non-equivalent. First experiments assessing the im-pact on code coverage are promising.