Policy gradient reinforcement learning method for discrete-time linear quadratic regulation problem using estimated state value function

In this paper, we propose a policy gradient reinforcement learning method which directly estimates the gradient of the state value function (V-function) with respect to a feedback coefficient matrix using measurable data and uses it for policy improvement. The proposed method can be applicable to the case where the state-action value function (Q-function) is difficult to estimate, and can update the policy in an effective direction for reducing the accumulated cost.