An Improved Exploration Method for Cooperative Multi-UAV Policy Learning with Sparse Rewards

This paper proposes a novel unmanned aerial vehicle (UAV) cooperative decision-making method based on deep reinforcement learning for striking targets protected by enemy air defense systems (ADSs), wherein an improved exploration technique is proposed to address the challenge of sparse extrinsic rewards and enhance policy optimization. Firstly, we construct a multi-UAV decision-making environment that incorporates the dynamic and fine-grained ADS model. Then, to encourage the coordinated exploration of UAVs, we measure the novelty of global observation based on the modified random network distillation (RND) method and employ it as shared intrinsic rewards for all UAVs. The modification of novelty measurement is introducing an image-like representation that tightly integrates spatial and temporal information of the situation as the input of RND. Finally, the experimental results demonstrate that the proposed method develops effective UAV cooperative striking policies and excels baseline methods.