Hilbert-Space Reduced-Rank Methods For Deep Gaussian Processes

A deep Gaussian process is a hierarchy of Gaussian processes where the process at each level is Gaussian given the process on the next level. In this paper, we recast special deep Gaussian processes as solutions of stochastic partial differential equations (SPDEs). Each of these SPDEs has parameters which are functions of the solutions to other SPDEs. To avoid solving SPDEs explicitly, we transform the SPDEs to finite-dimensional objects by truncating the underlying Hilbert space expansion. We then use a Markov chain Monte Carlo technique designed for function spaces to sample its posterior distribution. For a one-dimensional signal example, we show that the regression can offer discontinuity detection and smoothness constraints, which are competing with each other.