Multi-Scale Zero-Order Optimization of Smooth Functions in an RKHS

Consider the problem of optimizing a black-box function under the assumption that the function is Holder smooth and has bounded norm in the reproducing kernel Hilbert space associated with a given kernel. We propose the LP-GP-UCB algorithm which augments a Gaussian process surrogate model with local polynomial estimators of the function to construct a multi-scale upper confidence bound to guide the search for the optimizer. We provide high probability bounds on the cumulative regret in terms of the maximum information gain and smoothness parameters for the kernel. We then show that the Holder smoothness assumption is satisfied for several commonly used and practically relevant kernels—the Matern, rational- quadratic, γ-exponential, and piecewise-polynomial kernels—and obtain explicit regret bounds for them as a result. These regret bounds establish the near-optimality of LP-GP-UCB for these kernels and are also the first explicit bounds for many of them. Finally, we demonstrate the practical benefits experimentally.