Three-Class ROC Analysis—The Equal Error Utility Assumption and the Optimality of Three-Class ROC Surface Using the Ideal Observer
Xin He; Frey, E.C.
Medical Imaging, IEEE Transactions on
Volume 25, Issue 8, Aug. 2006 Page(s):979 - 986
Digital Object Identifier   10.1109/TMI.2006.877090
Summary:Previously, we have developed a decision model for three-class receiver operating characteristic (ROC) analysis based on decision theory. The proposed decision model maximizes the expected decision utility under the assumption that incorrect decisions have equal utilities under the same hypothesis (equal error utility assumption). This assumption reduced the dimensionality of the "general" three-class ROC analysis and provided a practical figure-of-merit to evaluate the three-class task performance. However, it also limits the generality of the resulting model because the equal error utility assumption will not apply for all clinical three-class decision tasks. The goal of this study was to investigate the optimality of the proposed three-class decision model with respect to several other decision criteria. In particular, besides the maximum expected utility (MEU) criterion used in the previous study, we investigated the maximum-correctness (MC) (or minimum-error), maximum likelihood (ML), and Nyman-Pearson (N-P) criteria. We found that by making assumptions for both MEU and N-P criteria, all decision criteria lead to the previously-proposed three-class decision model. As a result, this model maximizes the expected utility under the equal error utility assumption, maximizes the probability of making correct decisions, satisfies the N-P criterion in the sense that it maximizes the sensitivity of one class given the sensitivities of the other two classes, and the resulting ROC surface contains the maximum likelihood decision operating point. While the proposed three-class ROC analysis model is not optimal in the general sense due to the use of the equal error utility assumption, the range of criteria for which it is optimal increases its applicability for evaluating and comparing a range of diagnostic systems

» View citation and abstract

Log in by entering your IEEE Web Account Username and Password.

IEEE Communications Society members: If you subscribe to the IEEE Electronic Periodicals Package or IEEE Electronic Periodicals Package Plus, you must access your subscription at www.comsoc.org.

Check with your librarian, information professional, or system manager to determine if you need to log in. Please complete the online Technical Support Form if you need assistance.

Select the Purchase History link to access the document. You will have 5 Days after purchase to access the Full Text PDF. Please complete the online Technical Support Form if you need assistance.

• Search and access Abstract records free of charge
• Register for table of contents alerts
• Purchase Full Text PDF documents

» Learn more about subscription options or how to become an IEEE Member.