Measuring and evaluating maintenance process using reliability, risk, and test metrics

In analyzing the stability of a software maintenance process, it is important that it is not treated in isolation from the reliability and risk of deploying the software that result from applying the process. Furthermore, we need to consider the efficiency of the test effort that is a part of the process and a determinate of reliability and risk of deployment. The relationship between product quality and process capability and maturity has been recognized as a major issue in software engineering based on the premise that improvements in the process will lead to higher-quality products. To this end, we have been investigating an important facet of process capability-stability-as defined and evaluated by trend, change and shape metrics, across releases and within a release. Our integration of product and process measurement serves the dual purpose of using metrics to assess and predict reliability and risk and to evaluate process stability. We use the NASA Space Shuttle flight software to illustrate our approach.