A Systematic Review of Cost Reduction Techniques for Mutation Testing: Preliminary Results

This paper reports on results from a systematic review that characterizes the state-of-the-art on cost reduction for mutation testing. It analyzes the evolution of research on this topic and its underlying goals and techniques, and identifies metrics used to measure cost reduction. The mixed search strategy used automatic search, snowballing, and a survey of authors of primary studies. The analysis is based on a set of 165 peer-reviewed studies, from which 146 present either original or updated approaches and results for cost reduction of mutation testing. A list of 6 main goals for cost reduction is presented, and 22 techniques were identified. Historically, 18 metrics have been used to measure the gains and losses observed in experimental studies. In the last decade, substantial growth in the number of published studies was observed, particularly among techniques such as selective mutation, evolutionary algorithms, control-flow analysis, and higher order mutation.