Adaptive selective kernel learning algorithms

This paper studies kernel regression problems. The focus is on studying kernel algorithms that use the least squares criterion and developing methods so that the solution in the dual observation space intelligently chooses training examples. The least squares-support vector machine (LS-SVM) and variants have attracted researchers as the solution to nonlinear problems can be formulated as an optimization problem that involves finding a solution to a set of linear equations in the primal or dual spaces. A drawback of using the LS-SVM is that the solution is not sparse, but involves a solution to a set of linear equations in the dual space that is dependent on the number of observations. This paper discusses an on-line algorithm that selectively chooses to add and delete training observations. Through examples we show that this algorithm can outperform LS-SVM solutions that use a larger window of randomly trained examples.