ISR-MAPF: Integrated Search and Reinforcement Learning method for Multi-Agent Path Planning

Multi-agent Pathfinding (MAPF) problems typically involve multiple agents finding a set of conflict-free paths, and are usually solved in a centralized manner. In this work, we explore distributed scenarios where there is no central controller with complete information about agent locations and targets. And agents must sequentially determine their actions independently without full knowledge of the state of the environment. We will look for long-term strategies, which involve constantly assigning new goals to the agent as it reaches previous goals. To address this complex challenge, we propose an approach that combines two complementary approaches: heuristic search planning and reinforcement learning through strategy optimization. Planning is used to build and re-plan individual paths, and we have enhanced our planning algorithms with specialized techniques designed to prevent congestion and improve the throughput of the system. We also use reinforcement learning to develop collision avoidance strategies that effectively guide agents along their paths. These strategies are implemented in the form of neural networks that can be effectively trained without additional reward shaping or additional external planning. We evaluate our approach in a wide range of scenarios, comparing it to state-of-the-art solutions. The results show that our approach consistently outperforms both learnable and non-learnable competitors, demonstrating higher throughput and better ability to generalize to maps not seen before in the training phase.