A fast incremental Kernel Principal Component Analysis for learning stream of data chunks

In this paper, a new incremental learning algorithm of Kernel Principal Component Analysis (KPCA) is proposed for online feature extraction in pattern recognition problems. The proposed algorithm is derived by extending the Takeuchi et al.'s Incremental KPCA (T-IKPCA) that can learn a new data incrementally without keeping past training data. However, even if more than two data are given in a chunk, T-IKPCA should learn them individually; that is, in order to update the eigen-feature space, the eigenvalue decomposition should be performed for every data in the chunk. To alleviate this problem, we extend T-IKPCA such that an eigen-feature space learning is conducted by performing the eigenvalue decomposition only once for a chunk of given data. In the proposed IKPCA, whenever a new chunk of training data are given, linearly independent data are first selected based on the cumulative proportion. Then, the eigenspace augmentation is conducted by calculating the coefficients for the selected linearly independent data, and the eigen-feature space is rotated based on the rotation matrix that can be obtained by solving a kernel eigenvalue problem. To verify the effectiveness of the proposed IKPCA, the learning time and the accuracy of eigenvectors are evaluated using the three UCI benchmark data sets. From the experimental results, we confirm that the proposed IKPCA can learn an eigen-feature space very fast without sacrificing the recognition accuracy.