Ranking-based evaluation of regression models
Rosset, S.
Perlich, C.
Zadrozny, B.
IBM T. J. Watson Res. Center, Yorktown Heights, NY, USA;
This paper appears in: Data Mining, Fifth IEEE International Conference on
Publication Date: 27-30 Nov. 2005
On page(s): 8 pp.-
ISSN: 1550-4786
ISBN: 0-7695-2278-5
INSPEC Accession Number: 8857365
Digital Object Identifier: 10.1109/ICDM.2005.126
Current Version Published: 2006-01-03
Abstract
We suggest the use of ranking-based evaluation measures for regression models, as a complement to the commonly used residual-based evaluation. We argue that in some cases, such as the case study we present, ranking can be the main underlying goal in building a regression model, and ranking performance is the correct evaluation metric. However, even when ranking is not the contextually correct performance metric, the measures we explore still have significant advantages: They are robust against extreme outliers in the evaluation set; and they are interpretable. The two measures we consider correspond closely to non-parametric correlation coefficients commonly used in data analysis (Spearman's p and Kendall's r); and they both have interesting graphical representations, which, similarly to ROC curves, offer useful "partial" model performance views, in addition to a one-number summary in the area under the curve. We illustrate our methods on a case study of evaluating IT wallet size estimation models for IBM's customers.
Index
Terms
Available to subscribers and IEEE members.
References
Available to subscribers and IEEE members.
Citing Documents
Available to subscribers and IEEE members.
Cart
