The impact of costs of misclassification on software quality modeling

A software quality model can make timely predictions of the class of a module, such as not fault prone or fault prone. These enable one to improve software development processes by targeting reliability improvement techniques more effectively and efficiently. Published software quality classification models generally minimize the number of misclassifications. The contribution of the paper is empirical evidence, supported by theoretical considerations, that such models can significantly benefit from minimizing the expected cost of misclassifications, rather than just the number of misclassifications. This is necessary when misclassification costs for not fault prone modules are quite different from those of fault prone modules. We illustrate the principles with a case study using nonparametric discriminant analysis. The case study examined a large subsystem of the Joint Surveillance Target Attack Radar System, JS-TARS, which is an embedded, real time, military application. Measures of the process history of each module were independent variables. Models with equal costs of misclassification were unacceptable, due to high misclassification rates for fault prone modules, but cost weighted models had acceptable, balanced misclassification rates.