A Fast GP Regression Method Using Banded Sparsification of Inverse Covariance

Modeling and prediction using Gaussian processes (GPs) have widespread applications in machine learning, information processing, automatic control, biology, and other fields. However, GP regression requires a computational complexity of $O(N^{3})$. To overcome this issue, this paper proposes a fast GP regression method based on a banded sparsification of the inverse covariance matrix. It follows that we first propose an approximate inversion algorithm rooted in the Kullback-Leibler (KL) divergence, which exhibits approximately linear computational complexity. Furthermore, we demonstrate it is essentially a higher-order Gaussian Markov process (GMP) model and minimum mean square error (MMSE) estimation. Next, by transforming the inverse covariance matrix into a banded sparse structure, we derive a more efficient GP regression algorithm, of which the core is to employ a high-order GMP model to effectively approximate the general GP.