A Model-Free Iteration Algorithm for Markov Jump Linear Systems Based on Gauss-Seidel Method

This paper focuses on the linear quadratic regulator problem of discrete-time Markov jump linear systems without knowing the system matrices. A model-free fixed-point iteration algorithm is proposed to learn the optimal state feedback control law without the requirement of an initial admissible control policy. Analogous to the Gauss-Seidel method for linear equations, the model-free algorithm is constantly iterating with the latest information of each mode. It is proved that the algorithm converges monotonically to the optimal solution. In addition, our algorithm is faster than the classical model-based value iteration method. Finally, an example is used to illustrate our results.