Infinite-Dimensional Sums-of-Squares for Optimal Control

In this paper, we introduce an approximation method to solve an optimal control problem via the Lagrange dual of its weak formulation, which applies to problems with an unknown, non-necessarily polynomial, dynamics accessed through samples, akin to model-free reinforcement learning. It is based on a sum-of-squares representation of the Hamiltonian, and extends a previous method from polynomial optimization to the generic case of smooth problems. Such a representation is infinite-dimensional and relies on a particular space of functions – a reproducing kernel Hilbert space – chosen to fit the structure of the control problem. After subsampling, it leads to a practical method that amounts to solving a semi-definite program. We illustrate our approach numerically on a low-dimensional control problem.