Deep Reinforcement Learning with Local Communications for Continuous Pathfinding Problems in Multi-Agent Systems

We propose a multi-agent deep reinforcement learning (MADRL) method to efficiently solve continuous multi-agent path-finding (MAPF) problems, in which multiple agents iteratively reach their goals via collision-free paths. Various algorithm-based methods exist for MAPF; however, many are centralized or intrinsically sequential, limiting their applicability and increasing computational costs as the number of agents increases. Conversely, some MADRL-based methods enable distributed pathfinding using only local information, making their costs independent of the number of agents after training. However, these methods sometimes lack efficiency, and the quality of the learned behavior may decrease when the range of local information is expanded owing to nonessential data. Our proposed method, distributed pathfinding with centralized training (DPCT), allows agents to train a central network using locally relevant information through communication with only nearby agents to generate efficient moves toward their goals. During execution, the agents use their own networks, which are copies of the central network, to concurrently determine their moves. Our experiments demonstrate that our method achieves more efficient goal-directed moves than conventional methods, and finds reasonable solutions for continuous MAPF, even in previously unexperienced environments.