Decentralized Online Learning in RKHS With Non-Stationary Data Streams: Non-Regularized Algorithm

We propose a decentralized online non-regularized learning algorithm in reproducing kernel Hilbert space (RKHS) with non-stationary data streams over a network. The algorithm of each node in the network consists of an innovation term and a consensus term. The innovation term is to update the node's estimate by using its own measurement data, and the consensus term is a weighted sum of its own estimate and the estimates of its neighboring nodes. We prove that if the graph is connected, and online data streams satisfy the infinite-dimensional spatio-temporal persistence of excitation condition, then all nodes' estimates are both mean square and almost surely strongly consistent with the unknown function. Especially, for independent and non-identically distributed data streams, the algorithm is both mean square and almost surely strongly consistent if the average marginal probability measures during a finite-length period induced by the random data at each node converges in the dual of a Hölder space.