Input space versus feature space in kernel-based methods
Scholkopf, B.; Mika, S.; Burges, C.J.C.; Knirsch, P.; Muller, K.-R.; Ratsch, G.; Smola, A.J.
Neural Networks, IEEE Transactions on
Volume 10, Issue 5, Sep 1999 Page(s):1000 - 1017
Digital Object Identifier 10.1109/72.788641
Summary:This paper collects some ideas targeted at advancing our
understanding of the feature spaces associated with support vector (SV)
kernel functions. We first discuss the geometry of feature space. In
particular, we review what is known about the shape of the image of
input space under the feature space map, and how this influences the
capacity of SV methods. Following this, we describe how the metric
governing the intrinsic geometry of the mapped surface can be computed
in terms of the kernel, using the example of the class of inhomogeneous
polynomial kernels, which are often used in SV pattern recognition. We
then discuss the connection between feature space and input space by
dealing with the question of how one can, given some vector in feature
space, find a preimage (exact or approximate) in input space. We
describe algorithms to tackle this issue, and show their utility in two
applications of kernel methods. First, we use it to reduce the
computational complexity of SV decision functions; second, we combine it
with the kernel PCA algorithm, thereby constructing a nonlinear
statistical denoising technique which is shown to perform well on
real-world data
View citation and abstract |
Cart
