Saying, "I Am Testing," is Enough to Improve the Product: An Empirical Study

We report on an empirical study performed in an industrial environment in which the impact of modifying test practices is measured. First, the efficiency of the existing test practices is measured, by analyzing the software defects using a modified version of Orthogonal Defect Classification (ODC). The study was conducted in two phases lasting seven months each. In the first phase (Phase A), defects were recorded using existing implicit testing practices. Developers were then required to make those testing activities explicit based on a simple scheme. After a few months of employing this explicit testing practice, a new defect recording phase (Phase B), which also lasted seven months, was conducted. The impacts of explicit test practices are presented in terms of the ratios of defects found by the users and the nature of the testing activities performed. Results show that even a small improvement, such as requiring testing activities to be explicit, can have a measurable positive impact on the quality of the product.