Multi-Agent Proximal Policy Optimization with Spatial Attention for Scalable Unmanned Aerial Vehicle Swarm Control

In recent years, autonomous swarm navigation in dynamic, partially observable environments have presented important challenges for learning efficiency, coordination, and scalability. The traditional Multi-Agent Deep Deterministic Policy Gradient (MADDPG) suffer from sample inefficiency, weak adaptability and high communication requirements. To overcome these limitations, Multiagent Proximal Policy Optimization with Multiscale Spatial Attention (MAPPO-MSA) is proposed. Initially, a complex 3-Dimensional (3D) environment with both static and dynamic obstacles was simulated to provide each Unmanned Aerial Vehicle (UAV) with onboard sensors to perceive its local surroundings and make decisions based on partial local observations. Then, the reward function is designed to encourage reaching targets, avoiding collisions, energy-efficient trajectory planning, and swarm consistency. Subsequently, each agent is trained with a centralized critic accessing global information using MAPPO with independent actors depending on localized observations. Next, an MSA is incorporated to aid the agents' selective perception of nearby agents and environmental cues. Finally, the framework adopts a strategy of centralized training and decentralized execution to enhance real-world frameworks. The results show that MAPPO-MSA performs better than the existing methods in terms of energy consumption (55) and task completion time (120 seconds) respectively.