Automatic Collision Avoidance via Deep Reinforcement Learning for Mobile Robot

Collision avoidance is one of the most critical problems to be explored in the mobile robot field for finding an optimal path between source and destination. In this paper, we propose a novel DRL-based mapless collision avoidance algorithm for mobile robots, which maps the raw sensor data to control-level instructions and realizes robot navigation in an unknown environment without any complex numerical calculation. A game-like simulation environment enables the agent to interact with the surroundings and optimize its policy. The simulation environment is abstracted as walls and cylindrical obstacles, and it would be randomly initialized at the beginning of the training process to improve the generalization ability of DRL policy and obtain reliable test data for comprehensive analysis. We compare the performance of some state-of-the-art deep reinforcement learning algorithms with randomized environments, including DDPG, TD3, and SAC. The result shows that TD3 and SAC both gain a high passing rate when the obstacle density is from 0.0 to 0.10. The agent with DDPG fails to learn in the training environments and performs poorly. When obstacle density is 0.0 to 0.5, TD3 is more stable and performs better than SAC. However, SAC serves better when the obstacle density is 0.10 to 0.20. Both TD3 and SAC generate smooth trajectories in the test stage. The video and code are available: https://github.com/Vinson-sheep/turbot_rl.