DRL-UTPS: DRL-Based Trajectory Planning for Unmanned Aerial Vehicles for Data Collection in Dynamic IoT Network

Using highly maneuverable Unmanned Aerial Vehicles (UAV) to collect data is a fast and efficient method that is widely studied. In most studies, they assume that the UAVs can obtain the location of the Cluster Head (CH) before take-off, allocate CHs, and optimize the trajectory in advance. However, in many real scenarios, many sensing devices are deployed in areas with no basic communication infrastructure or cannot communicate with the Internet due to emergencies such as disasters. In this kind of sensing network, the surviving devices often change, and the CHs cannot be known and allocated in advance, thus bringing new challenges to the efficient data collection of the networks by using UAVs. In this paper, a UAV path planning scheme for IoT networks based on reinforcement learning is proposed. It plans hover points for UAV by learning the historical location of CHs and maximizes the probability of meeting CHs and plans the shortest UAV path to visit all hover points by using the simulated annealing method. In addition, an algorithm to search for the location of CHs is proposed which is named Cluster-head Searching Algorithm with Autonomous Exploration Pattern (CHSA-AEP). By using CHSA-AEP, our scheme enables the UAV to respond to the position change of the CHs. Finally, we compare our scheme with other algorithms (area coverage algorithms and random algorithm). It is found that our proposed scheme is superior to other methods in energy efficiency and time utilization ratio.