Assessment of fault-detection processes: an approach based on reliability techniques

Two major factors influence the number of faults uncovered by a fault-detection process applied to a software artifact (e.g., specification, code): ability of the process to uncover faults, quality of the artifact (number of existing faults). These two factors must be assessed separately, so that one can: switch to a different process if the one being used is not effective enough, or stop the process if the number of remaining faults is acceptable. The fault-detection process assessment model can be applied to all sorts of artifacts produced during software development, and provides measures for both the 'effectiveness of a fault-detection process' and the 'number of existing faults in the artifact'. The model is valid even when there are zero defects in the artifact or the fault-detection process is intrinsically unable to uncover faults. More specifically, the times between fault discoveries are modeled via reliability-based techniques with an exponential distribution. The hazard rate is the product of 'effectiveness of the fault-detection process' and 'number of faults in the artifact'. Based on general hypotheses, the number of faults in an artifact follows a Poisson distribution. The unconditional distribution, whose parameters are estimated via maximum likelihood, is obtained.