Safe and Efficient Auto-tuning to Cross Sim-to-real Gap for Bipedal Robot

Recent advances in both legged robot locomotion and Reinforcement Learning have shown a promising path for developing bipedal robot controllers. While the difference in dynamics between real world and simulation, also known as reality gap, still hinders the use. In this paper, we focus on sim-to-real bipedal robot locomotion task. We leverage the recent advances in auto-tuning sim-to-real transfer and use it to address sim-to-real bipedal robot locomotion problem. Similar to existing work, we first train a parameter searching model with dataset collected from simulator and use real-world data to tune the simulation parameters. However, the prediction tuning can be unreliable if the training dataset distribution fails to cover the real-world data. We address this problem by formulating this problem as an Out-of-distribution problem and further extending the current framework with a dataset verification model. With extended module, our method is capable of tuning the simulation parameters safely and efficiently. We demonstrate our method outperforms existing work and achieves sim-to-real bipedal robot locomotion on bipedal robot BITeno.