Visual-Based Deep Reinforcement Learning for Mobile Robot Obstacle Avoidance Navigation

To address the issue of navigation failure caused by light reflection in real-world navigation scenarios using inexpensive 2D LiDARs, traditional SAC-based algorithms face challenges such as inability to train in highly randomized and sparsely rewarded environments, as well as slow training. In this paper, we propose a combination of a monocular camera and a depth estimation model as a substitute for the inexpensive 2D LiDAR and introduce a variant algorithm called Sharing Encoder Self-Attention Soft Actor Critic (SESA-SAC) for collision-free indoor navigation of mobile robots. To improve the efficiency of robot learning in sparse environments, we collect expert data from 200 episodes and store them in a replay buffer. We conduct training by randomly sampling from both exploration data and expert data, without pre-training. To enhance training performance, we introduce a channel-wise self-attention structure and layer normalization in the network to learn better features. Additionally, we propose a shared feature extractor to achieve more stable training. Moreover, we conduct training and testing in GAZEBO, and the experimental results demonstrate that our proposed SESA-SAC algorithm outperforms traditional SAC algorithms in terms of convergence speed, stability, and efficiency for indoor navigation tasks.