An Effective Undersampling Approach to Deal with Class Imbalance Problem in Software Defect Prediction

Machine learning based software defect prediction (SDP) approaches assist in estimating where faults are likely to occur in source code. In real-life projects, fault-free modules are higher in number than faulty modules. This is referred to as class imbalance problem. Random over-sampling and random under-sampling (RUS) are the basic sampling strategies in data-level approaches for class imbalance problem. The RUS approach removes the instances randomly and due to randomness, loss of useful information may occur. On the other hand, over-sampling may lead to biased modeling. In this paper, we propose a new technique known as Structured Under-Sampling (SUS) to address both of these problems. The proposed approach systematically removes inconsistent and redundant instances. After that, most similar instances to the existing instances are removed to ensure minimum loss of information. For empirical evaluation, C4.5 classifier has been used to compare SUS with RUS and Tomek RUS while performance difference has been analyzed using F1-score and ROC. The results of our investigation show improved performance of SUS over other approaches. Moreover, it has been observed that the Imbalance ratio (IR) affects the performance of SUS. More specifically, when IR > 5 SUS outperformed in all results.