End-to-end Mobile Robot Autonomous Navigation via Deep Recurrent Q-network

In this paper, we propose a mobile robot autonomous navigation method based on a deep recurrent Q-network, which realizes the end-to-end mapping of the mobile robot’s perception of its behavior. In this study, we first extract features separately from the mobile robot’s own state information and the sensor’s detection information of the surrounding environment to form the current real-time observation. Then, the concatenated perception information is input into the recurrent neural network, and the perception representation of the current moment is enhanced by combining the historical state perception information to overcome the problem of bias in state evaluation under incomplete observable conditions. Finally, we design a step-wise reward function for the mobile robot’s autonomous navigation task based on the reward reshaping mechanism to guide the optimization direction of the strategy and further improve the convergence speed of the autonomous navigation model, achieving more efficient training. We constructed a 3D experimental environment to verify the effectiveness of the algorithm. The experimental results demonstrate the proposed method can effectively support the mobile robot’s autonomous navigation in complex scenarios, and compared with the contrast method, the advantages are obvious.