Regret Analysis of Learning-Based MPC With Partially Unknown Cost Function

The exploration–exploitation tradeoff is an inherent challenge in data-driven adaptive control. Though this tradeoff has been studied for multiarmed bandits (MABs) and reinforcement learning for linear systems, it is less well studied for learning-based control of nonlinear systems. A significant theoretical challenge in the nonlinear setting is that there is no explicit characterization of an optimal controller for a given set of cost and system parameters. We propose the use of a finite-horizon oracle controller with full knowledge of parameters as a reasonable surrogate to an optimal controller. This allows us to develop policies in the context of learning-based model-predictive control (MPC) and conduct a control-theoretic analysis using techniques from MPC and optimization theory to show that these policies achieve low regret with respect to this finite-horizon oracle. Our simulations exhibit the low regret of our policy on a heating, ventilation, and air-conditioning model with partially unknown cost function.