Dynamic Spanning Tree Coverage With Q-Learning: A Hybrid Framework for Efficient Robot Navigation in Environments With Moving Obstacles

This paper presents a novel hybrid framework that combines Dynamic Spanning Tree Coverage (D-STC) with Q-learning to address the challenges of mobile robot navigation and coverage in dynamic environments. The proposed method enables efficient exploration and complete area coverage while actively avoiding both static and moving obstacles using LIDAR-based real-time sensing and adaptive path planning. To evaluate robustness under varying levels of environmental dynamics, simulations were conducted in three obstacle speed scenarios: 1) obstacles slower than the robot, 2) equal-speed obstacles, and 3) faster obstacles. While D-STC provides structured traversal and Q-learning offers real-time adaptability, their integration significantly improves overall performance. In a $10 \times 10$ grid, the hybrid method required only 104, 109, and 114 steps across the respective scenarios, achieving the highest coverage efficiencies (99.0%, 97.0%, and 94.5%) and the lowest overlap rates (4.81%, 11.01%, and 17.11%). These results demonstrate that the proposed hybrid approach effectively reduces redundant traversal, enhances path efficiency, and dynamically responds to changing obstacle configurations—making it highly suitable for real-world, unpredictable, and obstacle-dense environments.