SCOUT: A Multi-objective Method to Select Components in Designing Unit Testing

The creation of a suite of unit testing is preceded by the selection of which components (code units) should be tested. This selection is a significant challenge, usually made based on the team member's experience or guided by defect prediction or fault localization models. We modeled the selection of components for unit testing with limited resources as a multi-objective problem, addressing two different objectives: maximizing benefits and minimizing testing cost. To measure the benefit of a component, we made use of metrics from static analysis (cost of future maintenance), dynamic analysis (risk of fault, and frequency of calls), and business value. We tackled gaps and challenges in the literature to formulate an effective method, the Selector of Software Components for Unit Testing (SCOUT). SCOUT provides an automated extraction of all necessary data followed by a multi-objective optimization process. SCOUT is a method able to assist testers in different domains, and the Android platform was chosen to perform our experiments, taking nine leading open-source applications as our subjects. SCOUT was compared with two of the most frequently used strategies in terms of efficacy. We also compared the effectiveness and efficiency of seven algorithms in solving a multi-objective component selection problem. Our experiments were performed under different scenarios, and reveal the potential of SCOUT in reducing the market vulnerability, compared to others approaches. To the best of our knowledge, SCOUT is the first method to assist in an automated way software testing managers in selecting components for the development of unit testing, combining static and dynamic metrics and business value.