Research on Beyond-Visual-Range Air Combat Decision Making Based on Improved Self-Play Deep Reinforcement Learning

As air combat technologies evolve, beyond-visual-range (BVR) operations have become the dominant mode of engagement. This paper addresses the autonomous decision-making challenges in BVR air combat by proposing an improved self-play deep reinforcement learning method. First, process-guiding rewards are introduced to mitigate the issue of reward sparsity commonly faced in BVR decision-making research. Second, an improved self-play approach with auxiliary strategies is proposed, enabling the main policy to identify and address its weaknesses more effectively. Simulation experiments validate the method's effectiveness, with the trained agent achieving an average win rate of 79% against rule-based expert strategy model and exhibiting behavior closely aligned with human experts. The results demonstrate that the proposed method enhances both offensive and defensive decision-making, ensuring robust performance in complex BVR scenarios.