Evaluating the Effectiveness of Conventional Machine Learning Techniques for Defect Prediction: A Comparative Study

Machine learning based techniques have been widely used in the literature for defect prediction. Although a number of comparative studies among different machine learning algorithms exist, these neither preprocess the data nor use valid metrics based on the quality of the data, which hamper the validity of the study. Moreover, how simple and conventional machine learning techniques perform in case of defect prediction has not been studied extensively in a valid way. This paper compares simple machine learning techniques for defect prediction on a systematically preprocessed data set, which is the popular NASA defect data set. Considering the quality of the data set, valid metrics have been used to statistically compare the performance of these algorithms. Moreover, the effect of feature selection is studied. It has been observed that these classifiers perform similarly for most of the data sets. Additionally, performing feature selection has been found helpful as it improves the overall accuracy of the defect prediction regardless of any learning algorithms used. The results also show the importance of data preprocessing and data quality for defect prediction.