Achieving Optimal Complexity Guarantees for a Class of Bilevel Convex Optimization Problems

We design and analyze a novel regularized accelerated proximal gradient method for a class of bilevel optimization problems where from the optimal solution set of a composite convex optimization problem, we seek to find a solution that minimizes a secondary convex objective function. When the optimal solution set of the lower-level problem admits a weak sharpness property, we significantly improve existing iteration complexity to O(E-
^0.5
) for both suboptimality and infeasibility error metrics, where ∊ > 0 denotes an arbitrary scalar. We also obtain guarantees in absence of the weak sharpness property. In addition, contrary to some existing methods that require solving the optimization problem sequentially (initially solving an optimization problem to approximate the solution of the lower-level problem followed by a second scheme), our method concurrently solves the bilevel optimization problem. To the best of our knowledge, the proposed algorithm achieves the best-known iteration complexity, which matches the optimal complexity for single-level convex optimization. Preliminary numerical experiments on a sparse linear regression problem validate the efficacy of our approach.