Efficient and Real-Time Reinforcement Learning for Linear Quadratic Systems with Application to H-Infinity Control

This paper presents a model-free, real-time, data-efficient Q-learning-based algorithm to solve the
$\mathrm{H}_{\infty}$
control of linear discrete-time systems. The computational complexity is shown to reduce from
$\mathcal{O}(\underline{q}^{3})$
in the literature to
$\mathcal{O}(\underline{q}^{2})$
in the proposed algorithm, where
$q$
is quadratic in the sum of the size of state variables, control inputs, and disturbance. An adaptive optimal controller is designed and the parameters of the action and critic networks are learned online without the knowledge of the system dynamics, making the proposed algorithm completely model-free. Also, a sufficient probing noise is only needed in the first iteration and does not affect the proposed algorithm. With no need for an initial stabilizing policy, the algorithm converges to the closed-form solution obtained by solving the Riccati equation. A simulation study is performed by applying the proposed algorithm to real-time control of an autonomous mobility-on-demand (AMoD) system for a real-world case study to evaluate the effectiveness of the proposed algorithm.