LiDAR-Based Autonomous Exploration Method of Mobile Robot Using Deep Reinforcement Learning in Unknown Environments

The autonomous exploration holds significant application value in tasks, such as mine exploration and environmental modeling, and personnel search and rescue, effectively boosting task efficiency. Learning-based methods are well-suited for these scenarios. Nevertheless, they suffer from low-learning efficiency and challenges in transferring from simulation to reality. To improve efficiency in the exploration process of mobile robots, a novel 3-D LiDAR-based autonomous exploration method using deep reinforcement learning (DRL) in unknown environments is proposed. In particular, a sparse informative graph based on the complementary holes is proposed, which serves as the input for our model. Furthermore, our model is built on the soft actor-critic (SAC) algorithm, integrating reinforcement learning with a graph-based and self-attention mechanism policy network. Moreover, an enhanced reward in our model is also employed to boost exploration capability. The proposed method was analyzed and validated in both simulation and real-world scenarios, demonstrating superior exploration performance (11.53% in trajectory length, 13.45% in makespan, and 30.77% in planning time) than the state-of-the-art (SOTA) method in a $200\times 180$ m self-established scenario. It also demonstrates that our method enhances the efficiency and robustness of autonomous exploration tasks.