Learning to Navigate in Unknown Environments Based on GMRP-N

Navigation in unknown environments can be formulated as POMDP which remains a challenging problem till now. In this paper, a grid map based random policy networks (GMRP-N) is proposed to explictly map the past observation to actions. The GMRP-N is a CNN based deep neural network, which first maps the unknown world and then make decision based on the map. High sample efficient imitation learning method is used to train GMRP-N. During training, uniform random unknown environments are generated and dijkstra algorithm is adopted as expert. A grid map based deterministic policy networks (GMDP-N) is also constructed as comparision in this paper. We find that the random policy is very key for successfull navigation in unknown environments, which can help the robot jumping out of the trap. After well trained, the success rate of navigation to target in random unknown enviroment is higher than 97.5% for GMRP-N which shows the GMRP-N is very valid. Finally the generalization ability is tested and the success rate remains high even though the environment is very different from the training environments.