Loading [a11y]/accessibility-menu.js
A novel dataset-similarity-aware approach for evaluating stability of software metric selection techniques | IEEE Conference Publication | IEEE Xplore

A novel dataset-similarity-aware approach for evaluating stability of software metric selection techniques


Abstract:

Software metric (feature) selection is an important pre-processing step before building software defect prediction models. Although much research has been done analyzing ...Show More

Abstract:

Software metric (feature) selection is an important pre-processing step before building software defect prediction models. Although much research has been done analyzing the classification performance of feature selection methods, fewer works have focused on their stability (robustness). Stability is important because feature selection methods which reliably produce the same results despite changes to the data are more trustworthy. Of the papers studying stability, most either compare the features chosen from different random subsamples of the dataset or compare each random subsample with the original dataset. These either result in an unknown degree of overlap between the subsamples, or comparing datasets of different sizes. In this work, we propose a fixed-overlap partition algorithm which generates a pair of subsamples with the same number of instances and a specified degree of overlap. We empirically evaluate the stability of 19 feature selection methods in terms of degree of overlap and feature subset size using sixteen real software metrics datasets. Consistency index is used as the stability measure, and we show that RF is the most stable filter. Results also show that degree of overlap and feature subset size do affect the stability of feature selection methods.
Date of Conference: 08-10 August 2012
Date Added to IEEE Xplore: 17 September 2012
ISBN Information:
Conference Location: Las Vegas, NV, USA

Contact IEEE to Subscribe

References

References is not available for this document.