PCE: Multi-Agent Path Finding via Priority-Aware Communication & Experience Learning

Multi-agent path finding (MAPF) is an essential issue for warehouse automation, where multiple agents plan collision-free paths from the start to goal positions. Reinforcement learning (RL) has been employed to develop partially observable distributed MAPF methods that can be scaled to any number of agents. However, existing RL-based MAPF methods still have some limitations in handling redundant information and avoiding deadlock, resulting in a low success rate or longer makespan. This paper proposes a Priority-aware Communication & Experience learning method (PCE), which combines RL with a novel priority-aware multi-agent communication and a new priority-aware deadlock experience replay to tackle this challenge. To be specific, our innovation encompasses two-fold. Our proposed communication mechanism aims to handle redundant information, which establishes a dynamic communication topology based on agents' priorities and proposes a two-head priority-aware graph attention network to aggregate information. In order to help the agent avoid deadlock, we prioritize the expert experience that solves the deadlock when performing experience replay. We conduct multiple simulation experiments on warehouse-like structured grid maps. Compared with the state-of-the-art RL-based MAPF methods, PCE performs significantly better with a higher success rate and lower makespan in small and large MAPF and higher average throughput in the lifelong MAPF, which can further improve the efficiency of warehouse automation. Finally, we validate PCE using three Turtlebot3-Burger robots, which shows that PCE can be applied in real warehouse automation scenarios.