Deep Kernel: Learning Kernel Function from Data Using Deep Neural Network

Kernel function implicitly maps data from its original space to a higher dimensional feature space. Kernel based machine learning algorithms are typically applied to data that is not linearly separable in its original space. Although kernel methods are among the most elegant part of machine learning, it is challenging for users to define or select a proper kernel function with optimized parameter settings for their data. In this paper, we propose a novel method called Deep Kernel that can automatically learn a kernel function from data using deep learning. The deep kernel is currently utilized in classification, and dimension reduction and visualization. For the classification task, we evaluate the deep kernel method by comparing its performance with the optimized Gaussian kernels, both using support vector machines as the decision model, on different types of datasets. The experimental results show that the proposed deep kernel method outperforms the traditional methods with Gaussian kernels on most of the data sets. For the dimension reduction and visualization task, the deep kernel is used along with kernel PCA. The results are also compared and contrasted with using the RBF kernel with multiple parameters. The deep kernel is shown to be more powerful in dimension reduction and visualization than the RBF kernel.