Estimation of classifier performance
Fukunaga, K.
Hayes, R.R.
Sch. of Electr. Eng., Purdue Univ., West Lafayette, IN;
This paper appears in: Pattern Analysis and Machine Intelligence, IEEE Transactions on
Publication Date: Oct 1989
Volume: 11,
Issue: 10
On page(s): 1087-1101
ISSN: 0162-8828
References Cited: 15
CODEN: ITPIDJ
INSPEC Accession Number: 3562020
Digital Object Identifier: 10.1109/34.42839
Current Version Published: 2002-08-06
Abstract
An expression for expected classifier performance previously
derived by the authors (ibid., vol.11, no.8, p.873-855, Aug. 1989) is
applied to a variety of error estimation methods and a unified and
comprehensive approach to the analysis of classifier performance is
presented. After the error expression is introduced, it is applied to
three cases: (1) a given classifier and a finite test set; (2) given
test distributions a finite design set; and (3) finite and independent
design and test sets. For all cases, the expected values and variances
of the classifier errors are presented. Although the study of Case 1
does not produce any new results, it is important to confirm that the
proposed approach produces the known results, and also to show how these
results are modified when the design set becomes finite, as in Cases 2
and 3. The error expression is used to compute the bias between the
leave-one-out and resubstitution errors for quadratic classifiers. The
effect of outliers in design samples on the classification error is
discussed. Finally, the theoretical analysis of the bootstrap method is
presented for quadratic classifiers
Index
Terms
Available to subscribers and IEEE members.
References
Available to subscribers and IEEE members.
Citing Documents
Available to subscribers and IEEE members.
Cart
