Estimating the Leave-one-out Error for Support Vector Regression

Tuning multiple parameters for support vector machines is usually done by minimizing some estimates of the generalization error. Leave-one-out error is an unbiased estimate of the true generalization error, but the computation of it is time costing. Inspired by the leave-one-out bound for classification presented by Joachim, a new bound on leave-one-out error of support vector regression is derived in this paper. After the solution of the optimization problem for support vector regression is obtained, the bound can be computed with very little additional work. Experiments on benchmark datasets illustrate its ability to estimate the generalization error. The bound can be used to tune hyperparameters for support vector regression.