A Comprehensive Investigation of Modern Test Suite Optimization Trends, Tools and Techniques

Software testing is an important but expensive activity of software development life cycle, as it accounts for more than 52% of entire development cost. Testing requires the execution of all possible test cases in order to find the defects in the software. Therefore, different test suite optimization approaches like the genetic algorithm and the greedy algorithm, etc., are widely used to select the representative test suite without compromising the effectiveness. Test suite optimization is frequently researched to enhance its competences but there is no study published until now that analyzes the latest developments from 2016 to 2019. Hence, in this article, we systematically examine the state-of-the-art optimizations' approaches, tools, and supporting platforms. Principally, we conducted a systematic literature review (SLR) to inspect and examine 58 selected studies that are published during 2016-2019. Subsequently, the selected researches are grouped into five main categories, i.e., greedy algorithm (seven studies), meta-heuristic (28 studies), hybrid (six studies), clustering (five studies), and general (12 studies). Finally, 32 leading tools have been presented, i.e., existing tools (25 tools) and proposed/developed tools (seven tools) along 14 platform supports. Furthermore, it is noted that several approaches aim at solving the single-objective optimization problem. Therefore, researchers should focus on dealing with the multi-objective problem, as multi-objective versions outperform the single-objective ones. Moreover, less attention has been given to clustering-based techniques. Thus, we recommend exploring the machine learning and artificial intelligence-based optimization approaches in the future. A broad exploration of tools and techniques, in this article, will help researchers, practitioners, and developers to opt for adequate techniques, tools, or platforms as per requirements.