Multi-Scale Graph Enhanced Reinforcement Learning for Conflict Resolution in Dense UAV Networks

Effective conflict resolution is crucial to ensure the safety of unmanned aerial vehicles (UAVs) in increasingly congested low-altitude airspace. However, the inefficiency in representing large-scale UAV information hinders the performance of existing learning-based methods. To overcome this challenge, we propose an enhanced graph-based reinforcement learning (GRL) approach that models multi-agent interactions relationships. Specifically, a novel multi-scale graph reinforcement learning (MS-GRL) approach is utilized to learn UAV avoidance strategies in a dense environment. MS-GRL utilizes the time-evolving intensity of local conflicts to evaluate the global attention weights for UAVs in conflict, while aggregating UAV observation data through graph embedding and requisite feature refinement. In addition, to adaptively limit the safety region of the action space while minimize deviations from the original trajectory, a safety-constrained maneuver strategy is proposed. Experimental results demonstrate that MS-GRL outperforms state-of-the-art GRL methods when there are up to 100 UAVs and multiple obstacles.