Learning of Quadruped Robot Motor Skills Based on Policy Constrained TD3

The agile movement of quadruped robot requires rich professional knowledge and tedious manual adjustment. However, reinforcement learning does not require any professional knowledge to enable the quadruped robot to learn better movement gait and skills. The TD3 algorithm is widely used in continuous motion control, but it tends to converge to boundary actions, resulting in inability to learn the optimal strategy and overfitting. Inspired by behavior cloning, this paper proposes a policy constrained TD3 algorithm(PC- TD3), which adds behavioral constraints during the policy update process of TD3, updates the policy in the direction of expected behavior, and reduces boundary actions. The experiment was conducted on the Pybullet platform. The experimental results show that the algorithm proposed in this paper can enable the quadruped Robot learning to learn the walking skills. Compared with other mainstream algorithms, the experiment shows that PC- TD3 has better performance.