Improvements in estimating software reliability from growth test data

John Musa's first book on Software Reliability Engineering advises the analyst to use the Musa Basic Law or the Musa-Okumoto Logarithmic Law to estimate failure intensity, depending on which provides the best fit to the data. He refered to the papers by Tractenberg and Downs as providing a foundation for these models. I propose using Musa's basic model in combination with an approach Duane and Codier used to estimate failure intensity (aka instantaneous failure rate) for software. Musa noted in his last book that the Basic law is optimistic in estimating residual errors and that the logarithmic law is pessimistic because it implies infinite errors. There is currently a problem in deciding which law to use for software, Basic or Logarithmic, other than which has a higher correlation coefficient. I recommend that Musa's basic law be used but that the line be drawn using a method that reflects the cumulative nature of the statistics. This approach is based on both Downs' paper and that presented by E. O. Codier at the 1968 Annual Symposium on Reliability which described how to draw the line for Duane growth plots. Codier argued that when reliability growth data is plotted: (1) “The latter points, having more information content, must be given more weight than earlier points” (2) “The normal curve fitting procedure of drawing the line through the `center of gravity' of all the points should not be used.” and (3) “Unless the data is exceptionally noisy, the best procedure is to start the line on the last data point and seek the region of highest density of points to the left of it.” With regard to Musa basic plots, the region of highest density would be to the right of the last point, not to the left of it. It should be noted here that the IEEE Recommended Practice on Software Reliability for the application of Duane states that “Least squares estimates for a and b of the straight line on log-log paper can be derived using recommended practice linear regression estimates”. But Codier's recommendations (above) have been shown to result in a more accurate measure of MTBF for hardware. This would be just as true for the application of Duane growth plo ts for software as for hardware. This paper shows even greater improvements when Codier's methods are applied to Musa's Basic model for Software. The early paper by Thomas Downs referred to the curved lines that are fitted to operational profile data as “convex”, not logarithmic, because there is no firm basis for calling the distribution logarithmic. There are examples in physics of exponential decay, as in the half life of a radioactive element, that follow a logarithmic curve, but no such mechanism exists to justify fitting a log function to software test data. The recommended method avoids defining the optimistic and pessimistic extremes of the curves as linear, logarithmic or any other specific shape. It simply draws a line that follows the changing slope of the points naturally as originally proposed by Codier. This paper develops a methodology for calculating failure intensity from the slope of the resulting line and from the cumulative failure rate at the final data point. It avoids the optimism of the Basic law and the pessimism of the Logarithmic law as well as the decision of which to use.