Towards Efficient Multi-UAV Air Combat: An Intention Inference and Sparse Transmission Based Multi-Agent Reinforcement Learning Algorithm

With the increasing utilization of Unmanned Aerial Vehicles (UAVs) in military operations, multi-UAV air combat has been emerging as one of the most important modes for future warfare. How to achieve intelligent cooperative maneuver policies subject to limited information sharing caused by the communication constraints among UAVs is crucial for winning air combat. In this paper, we formulate the communication-constrained multi-UAV air combat problem as a Markov game and propose a novel Sparse Inferred Intention Sharing Multi-Agent Reinforcement Learning (SIIS-MARL) algorithm for improving the winning rate of multi-UAV air combat. Our proposed algorithm contains the following designs: An intention inference module that enables each UAV to infer the intentions of teammates through the Theory of Mind (ToM) network for improved cooperation among teammates, and an attention-based sparse transmission mechanism which utilizes the inferred intentions and encoded embeddings to learn communication weights of teammates for enabling efficient sparsity in communication without causing performance penalty. Simulation results validate the effectiveness of our proposed algorithm as compared with existing work.