Robust and Efficient Walking of a Bipedal Humanoid Robot via Reinforcement Learning

Bipedal humanoid robots represent a crucial avenue in robotics development due to their adaptability to human-centric work environments, spanning industrial, service, and rescue sectors. The attainment of robust and energy-efficient bipedal walking in real-world scenarios has persistently challenged both industrial and academic research. Two primary approaches exist in robot locomotion control: traditional model-based methods heavily reliant on environmental factors, burdened by intricate modeling complexities, and lacking generalization capabilities. The potential for advancements in adaptive locomotion control, often impeded by complex modeling processes, can be significantly enhanced through the application of reinforcement learning (RL). This paper introduces a newly developed full-scale bipedal humanoid robot named Xiao-Man. A RL-based actor-critic network is designed to facilitate the robot's terrain-adaptive and efficient walking behavior. The control policy training process incorporates task rewards and auxiliary rewards to achieve robust and energy-efficient bipedal walking. To support this, we have curated a dataset based on joint actuation truth data and trained a joint actuator network to bridge the gap between expected torque and actual response torque.The results demonstrate that our trained control policy empowers the bipedal humanoid robot to achieve robust, energy-efficient bipedal walking and adaptability to complex terrains using solely proprioceptive information.