Queue Formation and Obstacle Avoidance Navigation Strategy for Multi-Robot Systems Based on Deep Reinforcement Learning

In contemporary society, the widespread application of robotics across various domains emphasizes the critical role of robotic systems in performing tasks that are too dangerous or complex for humans. However, individual robots often struggle with these intricate tasks, necessitating the collaboration of multi-robot systems. This study proposes a novel deep reinforcement learning (DRL)-based method for achieving queue formation, obstacle avoidance, and navigation in multi-robot systems. The approach includes a specialized state representation and reward mechanism tailored to queue formation, where the state design incorporates the relative positions and actions of robots in the queue to ensure they maintain formation and avoid obstacles. The reward mechanism encourages robots to adjust their speed and position, preserving the queue structure even in complex environments. To address the challenges associated with initial DRL training, we integrate curriculum learning, which facilitates smoother and faster convergence by gradually increasing task difficulty. This approach enables the system to efficiently complete the training process and enhance performance. Upon successful formation training, a speed control mechanism is introduced to improve the stability of the queue during navigation, ensuring robots maintain uniform speed and formation while avoiding obstacles. Simulation results demonstrate the effectiveness of the proposed method in managing queue formations across various robot numbers and environmental complexities. The results highlight robust performance with up to 10 robots, maintaining effectiveness in both static and dynamic environments. However, performance limitations are observed with larger fleets, indicating the need for further advancements in collision avoidance and speed control.