Sim-to-Real Reinforcement Learning for Local Mobile Robot Navigation in Dynamic Environments

Unprecedented agility and dexterous manipulation have been demonstrated with controllers based on deep reinforcement learning (RL), with a significant impact on legged and humanoid robots. Modern tooling and simulation platforms, such as NVIDIA Isaac Sim, have been enabling such advances. This article focuses on demonstrating the applications of Isaac in local planning and obstacle avoidance as one of the most fundamental ways in which a mobile robot interacts with its environments. The main contribution of the paper is to provide an open-source approach to learning end-to-end local navigation that generalizes across mobile robots, with additional material providing a tutorial-like experience. The paper demonstrates that state-of-the-art controllers can be trained, by benchmarking with the de-facto ROS 2 standard, Nav2. We additionally discuss the sim-to-real process. Our experiments in simulation and in the real-world demonstrate safe and efficient navigation in the presence of both static and dynamic obstacles. A step-by-step appendix tutorial is available at RL-Navigation/appendix.pdf and the open-source code is available at https://github.com/sahars93/RL-Navigation.