Actor-Critic Method to Solve the Linear Quadratic Problem of Markov Jump Linear System

This paper investigates learning linear quadratic regulator of Markov jump linear system(MJLS). Firstly, we propose an estimation method for state-action value function(Q-function) based on Bellman residual equation. This method can be seen as solving a stochastic optimization problem. Secondly, we propose a single time scale actor-critic(AC) method to solve the optimal control problem. It is based on the natural gradient descent and estimation method of Q-function we proposed. Finally, we analyze the convergence rate of AC method when the estimation of gradient is accurate and inaccurate respectively.