Research on UAV Coverage Search Based on DDQN in Unknown Environments

The utilization of unmanned aerial vehicle (UAV) for area coverage search is highly sought after in both military and civil domains, including but not limited to traversal search, mission reconnaissance, patrol detection, wildfire suppression control, remote sensing mapping, agricultural preservation, and accident search and rescue. This paper focuses on the problem of area coverage search for a single UAV in environments with the presence of unknown dynamic and static targets as well as hazardous areas. Here the UAV only knows the state of a small area and remembers the actions of the last time step without long-term memory. The objective is to design an adaptive and transferable algorithm for the UAV to find all static and dynamic targets with the minimum path repetition rate in the task area where both dangerous areas and completely unknown target information exist. Because the UAV has limited ability to observe all the map information, a partially observable Markov decision process is first formulated. Then we develop a coverage search algorithm based on Double Deep Q Network (DDQN) with the help of curriculum learning. By designing multiple constraint reward functions and employing path repetition rate and target batting average as evaluation metrics, the proposed algorithm facilitates the rapid adaptation of UAVs to diverse task environments. Simulation environment and algorithm models are finally established to illustrate the efficacy of the algorithm, which shows that the proposed algorithm with curriculum learning has rapid convergence, minimal path redundancy, high target acquisition rate, robust portability, and adaptability to variations in map area, hazard zones, and target quantity.