Reinforcement Learning Combined with Heuristic Search for Solving Discrete Space Path Planning Problems

Reinforcement learning (RL) has been successfully applied to solve path planning problems, but learning is generally slow. The main reason is not making full use of information collected during interaction with the environment. This paper proposes a novel method to solve the discrete space path planning problem in an environment without prior knowledge with intensive obstacles based on RL and heuristic search. Firstly, we apply Dyna-Q algorithm of RL to explore the map and search for the target point and optimize its policy with upper confidence bound (UCB). Then, when the target point is found, we use heuristic search to plan the path from the starting point to the target point and narrow the path to a small range. Finally, we combine Dyna-Q algorithm with the heuristic search recommended path for path planning. We evaluate our algorithm using maze navigation problem. The results verify that heuristic search accelerates Dyan-Q convergence.