Two-Stage Reinforcement Learning Conflict Resolution Algorithm for Multi-UAV System Based on RGCR Pre-Training

As Unmanned Aerial Vehicles (UAV s) are increasingly deployed in both civil and military domains, it is common for multiple UAV s to operate within shared airspace. Therefore, effective conflict resolution between UAVs is crucial for ensuring operational safety. While reinforcement learning (RL) has the potential to address conflict resolution in multi-UAV scenarios, it faces challenges such as high variance and low reproducibility due to the absence of a fixed training set with real labels. To address these challenges, this paper presents a novel two-stage deep reinforcement learning-based approach for conflict resolution. In the first stage, the Reciprocal Geometric Conflict Resolution (RGCR) method is employed to generate a robust dataset for conflict resolution, which is then used to pre-train the action network in a supervised manner. In the second stage, the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm is applied for reinforcement learning, where the conflict resolution policy is optimized by maximizing a reward function to balance the parameters of the dual networks. The proposed method is compared with other algorithms in multi-UAV convergence scenarios. Simulation results demonstrate the effectiveness of the pre-training stage and highlight the robustness and generalization capability of the proposed approach.