Nash equilibrium of Aerocraft Pursuit-evasion Games Based on Self-Play strategy

This study addresses the pursuit-evasion game between two Aerocraft in three-dimensional space, taking into account state and control constraints, with the objective of ascertaining the optimal control strategies for both the pursuer and evader. The pursuit-evasion game is initially decomposed into two distinct optimization problems, which simplifies the intricate dynamics at play. An iterative self-play technique is then devised to progressively refine the control policies through repeated simulations of the game scenario, enabling the system to learn and adapt over successive iterations. When the Hamiltonian constructed using the Pontryagin Maximum Principle (PMP) has a unique solution, the iterative policy can converges to the Nash equilibrium. However, the nonlinear equations derived from PMP makes it difficult to determine the control policy. Additionally, the system dynamics considered in this study involve a high-dimensional state vector with constraints, further complicating the problem. To tackle these challenges, we transform the optimal control problem into a nonlinear programming problem using the self-play method. Finally, the numerical case is presented and analyzed to demonstrate the effectiveness of the proposed method.