Scalable, Federated Gaussian Process Training for Decentralized Multi-Agent Systems

Gaussian process (GP) training of kernel hyperparameters still remains a major challenge due to high computational complexity. The typical GP training method employs maximum likelihood estimation to solve an optimization problem that requires cubic computations for each iteration. In addition, GP training in multi-agent systems requires significant amount of inter-agent communication that typically involves sharing of local data. In this paper, we propose scalable optimization algorithms for federated training using Gaussian processes (GPs) in decentralized multi-agent systems. To decentralize the implementation of GP training with maximum likelihood estimation, we employ the alternating direction method of multipliers (ADMM). We provide a closed-form solution of the decentralized proximal ADMM for the case of GP hyperparameter training using the separable squared exponential kernel. Federated learning is promoted in decentralized networks by prohibiting local data exchange between agents. Moreover, we extend a centralized GP training method by augmenting local datasets to improve the GP training estimation accuracy with large-scale multi-agent systems. The efficiency of the proposed methods is illustrated with numerical experiments.