Enhancing Locomotion Abilities of Humanoid Robots Using Universal Gait Templates for Reinforcement Learning

Humans have remarkable physical abilities and neurological coordination, allowing them to perform an extensive variety of movements, most of which are cyclic or quasi-cyclic. Numerous methods have been proposed to achieve cyclic movements in humanoid robots, such as central pattern generators, model predictive control, imitation learning, and reinforcement learning. However, these methods mainly achieve a suboptimal single-gait or a restricted range of gaits, constraining the humanoid’s movement capabilities. To address this issue, we proposed a universal gait template generation method. It allows continuous variation in gait parameters, such as cycle period, swing time ratio, body height, and maximum foot elevation. Through sampling within this space, various gaits can be generated, such as standing, walking, speed walking, running, and jumping. Deep reinforcement learning is employed to track the generated randomized gait template as reference trajectories within a single network. This method allows the humanoid to achieve infinitely many physically feasible gaits and select the optimal one. We conducted simulation analyses and achieved robust control with zero-shot sim-to-real transfer on the Q-series humanoid robots’ hardware platform, achieving a maximum speed of 2 m/s, payload capacity of 20 kg, and a maximum impact resistance of 80 Ns.