Analyzing inspection data for heuristic effectiveness

A significant body of knowledge concerning software inspection practice indicates that the value of inspections varies widely both within and across organizations. Inspection effectiveness and efficiency may be affected by a variety of factors such as inspection planning, the type of software, the developing organization, and many others. In the early 1990's a governmental organization developing complex and highly critical software systems formulated heuristics for inspection planning based on best practices and their early inspection data. Since the development context at the organization has changed in some ways since the heuristics were proposed, it is important to assess whether the heuristics are still a suitable guideline to use. To investigate this question, we statistically evaluated the differences in effectiveness and efficiency between inspections that adhered to the heuristics and ones that did not. Our analysis revealed no significant difference in effectiveness or efficiency for most heuristics. We also learned that compliance with the heuristics is diminishing over time.