The First-Degree Stochastic Dominance Rule of Comparing Pattern Recognition Algorithms with Accuracy Metric

Accuracy of pattern recognition algorithm is an uncertain metric and should be transformed to utility value for better understanding of the performance of recognition algorithm. According to the uncertainty of accuracy and its utility transforming demands, we use stochastic dominance to compare two recognition algorithms. The first-degree dominance (FSD) rule is checked and proved to be unaffected by any non-diminishing utility transformation. The sufficient and necessary condition of the FSD rule is not always satisfied in the whole value range of accuracy, so we analyze its establishing condition firstly. The existing scope of the FSD rule is given by explicit expression and then some simplified cases are discussed. Both theorical analysis and numerical examples show that the FSD rule can be applied directly in the conditions of using the same test set to compare two recognition algorithms. While in the other conditions of using different test sets, the FSD rule is still useful in most cases. The FSD rule gives us much flexibility in preparing test data and more confidence about the comparison results.