A h-D3QN-QMIX Design for Formation Decision in Air Combat

This paper addresses the formation decision problem in air combat. Based on the hierarchical reinforcement learning framework, a novel h-D3QN-QMIX algorithm is designed, where the D3QN term is used to determine the optimal formation according to the air combat state and at the same time the QMIX term is employed to achieve formation. The reward function is divided into the intrinsic and extrinsic rewards, where the former is designed based on the target position of the formation and the latter is designed based on the expected formation and combat results. It is showing that the win rate of the h-D3QN-QMIX is 71.4%, which is much higher than it of QMIX and rule-based method.