A Second Replicated Quantitative Analysis of Fault Distributions in Complex Software Systems

Background: Software engineering is searching for general principles that apply across contexts, for example, to help guide software quality assurance. Fenton and Ohlsson presented such observations on fault distributions, which have been replicated once. Objectives: We aimed to replicate their study again to assess the robustness of the findings in a new environment, five years later. Method: We conducted a literal replication, collecting defect data from five consecutive releases of a large software system in the telecommunications domain, and conducted the same analysis as in the original study. Results: The replication confirms results on unevenly distributed faults over modules, and that fault proneness distributions persist over test phases. Size measures are not useful as predictors of fault proneness, while fault densities are of the same order of magnitude across releases and contexts. Conclusions: This replication confirms that the uneven distribution of defects motivates uneven distribution of quality assurance efforts, although predictors for such distribution of efforts are not sufficiently precise.