Batched Stochastic Bandit for Nondegenerate Functions

This paper studies batched bandit learning problems for nondegenerate functions. Over a compact doubling metric space (X,D), a function f : X → R is called nondegenerate if there exists L ≥ λ > 0 and q ≥ 1, such that λ (D(x, x*))^q ≤ f(x) − f(x*) ≤ L(D(x, x*))^q, x ∈ X, where x* = arg min_z∈X f(z) is the unique minimizer of f over X. In this paper, we introduce an algorithm that solves the batched bandit problem for nondegenerate functions nearoptimally. More specifically, we introduce an algorithm, called Geometric Narrowing (GN), whose regret bound is of order O(A^d₊ √T), where d is the doubling dimension of (X,D), and A₊ is a constant independent of d and the time horizon T. In addition, GN only needs O(log log T) batches to achieve this regret. We also provide lower bound analysis for this problem. More specifically, we prove that over some (compact) doubling metric space of doubling dimension d: 1. For any policy π, there exists a problem instance on which π admits a regret of order Ω (A^d₋ √T), where A₋ is a constant independent of d and T; 2. No policy can achieve a regret of order A^d₋ √T over all problem instances, using less than Ω(log log T) rounds of communications. Our lower bound analysis shows that the GN algorithm achieves near optimal regret with minimal number of batches.