Autonomous UAV Maneuvering Decisions by Refining Opponent Strategies

In a typical game scenario, the attention in unmanned aerial vehicle (UAV) air combat should be focused on both sides' maneuvering decision strategies. However, most existing studies focus only on improving their own maneuvering decisions, ignoring the importance of the opponent's strategy in the two-player game. This article proposes a reinforcement learning (RL) based air combat decision method considering the opponent's maneuvering strategy. Through the proposed limited imitation offline RL (LIORL) method, the opponent's air combat decision method is refined using existing air combat data. Based on the enemy's superior strategy, an air combat simulation environment is created, and the RL method is applied to train the agent for UAV air combat maneuvering decision strategies. No current research utilizes static datasets for the acquisition of adversary strategies. This study, in contrast, offers a more reliable approach in comparison with methodologies predicated on assuming adversary strategies. Ablation experiments have been meticulously executed to showcase the capability of the LIORL algorithm in identifying the optimal enemy strategy while exerting a lesser impact on the dataset than incumbent algorithms. The employment of the LIORL algorithm enables the agent to refine the adversary's strategy, resulting in an augmented win rate against our UAV. Through air combat simulations, we empirically validate the agent's decision-making process based on the adversary's strategy, thereby affirming the efficacy of the proposed methodology.