Safe Sim-to-Real Robot Exploration with Constrained Bayesian Optimization

When tuning parameters of robot systems, it is common but undesirable to encounter failure. In practice, a natural approach is to evaluate the policy in advance using simulation. However, challenges arise due to the reality-gap and the difficulty in automating the tuning process. In this work, we propose a framework that leverages prior knowledge from biased simulator for safe exploration. With Bayesian optimization, our method first samples in simulator until the candidate of optima stabilizes. Then, a cautious exploration based on the prior is executed in real-world to obtain more information. The above actions run iteratively and automatically through our two-step strategy and source-switching rule. We evaluate the resulting framework on synthetic experiments and a real trajectory tracking task on unmanned surface vessel.