Reinforcement Learning of Aerial Combat Maneuver Decisions Based on UAVs Situation

Traditional UAV air warfare decision methods lack innovative maneuvering behavior. In this paper, we proposed an autonomous decision-making for UAVs based on deep reinforcement learning to solve the maneuvering decision-making problem in air warfare. Firstly, the air combat situation can be effectively calculated via Bayesian inference theory. Then, the UAV state transfer equation is established based on the its three-degree-of-freedom model. Specifically, the aerial combat situation affects the distribution of rewards. And the rewards are comprehensively designed by angle, distance, speed, and altitude. Thus, with Bayesian inference, the situation is calculated for changing the weights of rewards. Besides, the Ornstein-Uhlenbeck (OU) random noise, is introduced to improve the algorithm's exploratory performance during the training process. The aerial combat simulation results demonstrate the effectiveness of the proposed method, with the moving average reward values ultimately converging within the range of 120-130.