MPC-Guided Reinforcement Learning for Enhanced Bipedal Walking Stability on Uneven Terrain

Bipedal robots have great application potential in many task scenarios, but their lack of robust locomotion capabilities on uneven terrains significantly limits their practical use. Based on this, this paper proposes a motion control framework that integrates model predictive control (MPC) with reinforcement learning (RL) to generate high-quality, periodic, and energy efficient gaits, as well as to enhance the stability of bipedal robots traversing uneven terrain. First, a more expressive kinodynamics model is utilized to enhance the diversity of motion patterns generated by MPC. Subsequently, RL is employed to adjust the reference trajectories produced by MPC. The modelbased MPC improves the physical safety and generalization ability of the behavior policies, while RL compensates for MPC’s difficulty in dealing with unknown disturbances. Experimental results indicate that the proposed method generates periodic, natural, and energy-efficient gaits, showcasing excellent speed tracking ability and adaptability to unknown environments. Additionally, the guidance provided by MPC reduces the difficulty of reward shaping for RL.