Motion Sequence Learning for Robot Walking Based on Pose optimization

Deep reinforcement learning provides a model-free method to teach a bipedal robot to walk, which is always a challenging task. However, low convergence and training efficiency limit the applications. To overcome such limitations, in this work an effective framework combined with traditional controllers and reinforcement learning is proposed, based on the realistic robot model Robotis-op3. The objective is to realize the motion sequence learning based on pose optimization (MSLPO). The Q-learning algorithm is used to train the walking parameters of the traditional model-based controller to get optimized poses. Based on pose optimization, deep reinforcement learning is utilized to train the walking controller. By combining pose optimization and reinforcement learning with reward shaping, the effective controller is achieved that can make the robot walk in a stable and fast manner. The proposed controller attains over 468% improvement in velocity and 51.5% decrease in relative deviation compared with the traditional controller. The proposed framework can also significantly improve the training efficiency compared with a continuous reinforcement learning algorithm. The proposed model is implemented in a simulated environment Pybullet and tested in different terrains. This work can be extended to other tasks and it is independent of a particular robot model.