Optimization of Restricted ROC Surfaces in Three-Class Classification Tasks
Edwards, D.C.; Metz, C.E.
Medical Imaging, IEEE Transactions on
Volume 26, Issue 10, Oct. 2007 Page(s):1345 - 1356
Digital Object Identifier 10.1109/TMI.2007.898578
Summary:We have shown previously that an N-class ideal observer achieves the optimal receiver operating characteristic (ROC) hypersurface in a Neyman-Pearson sense. Due to the inherent complexity of evaluating observer performance even in a three-class classification task, some researchers have suggested a generally incomplete but more tractable evaluation in terms of a surface, plotting only the three ldquosensitivities.rdquo More generally, one can evaluate observer performance with a single sensitivity or misclassification probability as a function of two linear combinations of sensitivities or misclassification probabilities. We analyzed four such formulations including the ldquosensitivityrdquo surface. In each case, we applied the Neyman-Pearson criterion to find the observer which achieves optimal performance with respect to each given set of ldquoperformance description variablesrdquo under consideration. In the unrestricted case, optimization with respect to the Neyman-Pearson criterion yields the ideal observer, as does maximization of the observer's expected utility. Moreover, during our consideration of the restricted cases, we found that the two optimization methods do not merely yield the same observer, but are in fact completely equivalent in a mathematical sense. Thus, for a wide variety of observers which maximize performance with respect to a restricted ROC surface in the Neyman-Pearson sense, that ROC surface can also be shown to provide a complete description of the observer's performance in an expected utility sense.
View citation and abstract |
Cart
