3D ROC Histogram: A New ROC Analysis Tool Incorporating Information on Instances

While Receiver Operator Characteristic (ROC) curves have been a standard tool in the design and evaluation of binary classification problems, they have sometimes been blamed for ignoring some vital information in the evaluation process, such as predicted scores and the amount of information about the target that each instance carries. In this paper, a new classification performance method denoted as 3D ROC histogram is proposed for extending ROC curves into 3D space. In this histogram, the x-axis and the y-axis are respectively labeled as false positive rate, and true positive rate which are the same with traditional ROC space. The z-axis serves as a quantitative index that represents vital information, and the volume of the 3D ROC histogram (V3RH) acts as a summary index. The proposed method preserves merits such as robustness with respect to class imbalance and threshold independence, and also, it provides an easy way for incorporating additional information in the evaluation process. Experiments on real-world datasets were conducted, with results that confirmed it to be a reliable measure.