Almost Surely $\sqrt{T}$ Regret for Adaptive LQR

The linear-quadratic regulation (LQR) problem with unknown system parameters has been widely studied, but it has remained unclear whether $\tilde{ \mathcal {O}}(\sqrt{T})$ regret, which is the best known dependence on time, can be achieved almost surely. In this article, we propose an adaptive LQR controller with almost surely $\tilde{ \mathcal {O}}(\sqrt{T})$ regret upper bound. The controller features a circuit-breaking mechanism, which falls back to a known stabilizing controller when the input signal or control gain is large, thereby circumventing potential safety breach and ensuring the convergence of the system parameter estimate. Meanwhile, the circuit-breaking is shown to be triggered only finitely often, and hence has a negligible effect on the asymptotic performance of the controller. The proposed controller is also validated via simulation on Tennessee Eastman process (TEP), a commonly used industrial process example.