Path Planning of Autonomous Mobile Robot in Comprehensive Unknown Environment Using Deep Reinforcement Learning

In real-world situations, some unavoidable problems like significant dependence on environment information, long inference time and weak anti-disturbance ability are often involved in path planning of autonomous mobile robot (AMR) under unknown environments. To solve these issues, this article proposes an improved deep reinforcement learning-based path-planning algorithm to find out an optimized path for a class of AMRs. First, the path planning of AMR is described as a Markov decision process framework, and the double deep
$Q$
network (DDQN) is utilized to obtain the optimal adaptive solutions of AMR’s path planning. Second, a comprehensive reward function integrated with heuristic function is designed to navigate the AMR into the target area. Afterwards, an optimized deep neural network with an adaptive
$\varepsilon $
-greedy action selection policy is designed to deal with the tradeoff between exploration and exploitation, thus further to improve the global searching capability and the convergence performance for the AMR path planning. Moreover, Bezier curve theory is utilized to smooth the planned path. Finally, the comparative simulations are carried out to validate our proposed path-planning algorithm. The results show that, compared with DQN, A*, RRT, and APF algorithms, our improved DDQN algorithm can produce safer and shorter global paths in comprehensive unknown environments. Meanwhile, the IDDQN algorithm has strong adaptability to random disturbances in unknown environments.