Least Square Reinforcement Learning for Solving Inverted Pendulum Problem

Inverted pendulum is one of the classic control problem that could be solved by reinforcement learning approach. Most of the previous work consider the problem in discrete state space with only few exceptions assume continuous state domain. In this paper, we consider the problem of cart-pole balancing in the continuous state space setup with constrained track length. We adopted a least square temporal difference reinforcement learning algorithm for learning the controller. A new reward function is then proposed to better reflect the nature of the task. In addition, we also studied various factors which play important roles in the success of the learning. The empirical studies validate the effectiveness of our method.