Deep Reinforcement Learning Based Mobile Robot Navigation in Crowd Environments

Robots are becoming popular in assisting humans. The mobile robot navigation in human crowd environments has become more important. We propose a deep reinforcement learning-based mobile robot navigation method that takes the observation from the robot's onboard Lidar sensor as input and outputs the velocity control to the robot. A customized deep deterministic policy gradient (DDPG) method is developed that incorporates guiding points to guide the robot toward the global goal. We built a 3D simulation environment using an open dataset of real-world pedestrian trajectories that were collected in a large business center. The neural network models are trained and tested in such environments. We compare the performance of our proposed method with existing algorithms that include a classic motion planner, an existing DDPG method, and a generative adversarial imitation learning (GAIL) method. Using the measurement metrics of success rate, the number of times freezing, and normalized path length, extensive simulation results show that our method outperforms other state-of-the-art approaches in both trained and untrained environments. Our method has also better generalizability compared with the GAIL method.