Optimal Control Inspired Q-Learning for Switched Linear Systems

This paper studies Q-learning for quadratic regulation problem of switched linear systems. Inspired by the analytical results from classical model-based optimal control, a structured Q-learning algorithm is developed. The proposed algorithm consists of a carefully designed parametric approximator that respects the analytical structure of the exact Q-function and an associated parameter training algorithm. Based on a geometric insight gained from the analysis of the exact Q-function structure, training of approximation parameters is formulated as a matrix identification problem. Probabilistic guarantee on successful identification of all matrices using the proposed algorithm is rigorously proved under moderate conditions. Several numerical studies are conducted to demonstrate the effectiveness of the overall proposed Q-learning algorithm.