A Learning Approach to Derive Sparse Kernel Minimum Square Error Model

Kernel minimum square error (KMSE) model is computationally more tractable than other nonlinear methods, but it still has some drawbacks in theory and computational problems. Moreover, the characteristic that the classification efficiency of KMSE decreases as the size of the training sample set increases makes KMSE yield low classification efficiency for classification problems with a large number of training samples. In this paper, several methods which are developed for improving the classification efficiency of KMSE are assessed and their shortcomings are indicated. Then, KMSE is presented as a regression model. Taking advantage of local ridge regression, we develop an efficient KMSE classification technique. The proposed technique can sufficiently exploit the theoretical merit of local ridge regression which may produce more stable estimates with smaller variance than the least square error technique. This technique can also determine local regularization parameters properly and automatically, and then construct an improved KMSE model with lower structure complex which leads to a more efficient classification process. Experiments show that the improved KMSE model not only classifies much more efficiently but also obtains higher classification accuracy than KMSE, while outperforming several existing improved KMSE models.