Spatio-Temporal Graph Learning for Enhanced Agent Collaboration in Multi-Aircraft Combat

In recent years, significant progress has been made in Multi-Agent Deep Reinforcement Learning (MADRL) for addressing cooperative decision-making challenges in multi-aircraft air combat tasks. This paper introduces a novel Spatio-Temporal Relationship Graph Structure Learning method (STRGSL), aimed at overcoming the challenges in capturing the complex and dynamic interactions between agents. The proposed STRGSL constructs a historical behavior graph based on past observations as well as a real-time interaction graph from current observations, providing a comprehensive consideration of both immediate and long-term agent relationships. Leveraging a Graph Neural Network (GNN), STRGSL generates agent representations that fuse current and historical relationships. By integrating STRGSL into a MADRL framework, we jointly optimize both the structure of relationship graphs and the cooperative policies of agents. Experiments carried out in an aircraft combat scenario and two multi-agent cooperative scenarios demonstrate that the proposed STRGSL promotes collaboration among multiple agents, thereby enhancing the overall performance across different scenarios.