Two-Hop Packet Scheduling, Resource Allocation, and UAV Trajectory Design for Internet of Remote Things in Air–Ground Integrated Network

Compared with terrestrial network, the air–ground integrated network consisting of unmanned aerial vehicles (UAVs) and high-altitude platforms (HAPs) offers the advantages of large coverage, high capacity, and seamless connection. Therefore, the air–ground integrated network can provide effective communication services for the Internet of Remote Things (IoRT). In order to reduce the end-to-end (e2e) packet delay and avoid network congestion of the two-hop network, we investigate a joint packet scheduling, resource allocation, and UAV trajectory design problem, with the objective of minimizing the average packet queue delay from HAP to IoRT devices in the air–ground integrated network. This problem is nonconvex and difficult to solve by the traditional methods. In order to solve this problem, we reformulate it into a Markov decision process (MDP) first. And then, considering there are continuous and discrete hybrid action spaces in the MDP, we separate the primal action spaces into two subaction spaces, and utilize the basic idea of multiagent deep deterministic policy gradient (MADDPG) and multiagent double deep Q network (MADDQN) methods to solve them, respectively. After that, in order to improve the stability, convergence rate and learning efficiency, we introduce the basic idea of adaptive prioritized experience replay (PER), and propose a hybrid MADDPG-adaptive PER (MADDPG-APER) algorithm. Simulation results show that the proposed algorithm can reduce the average packet queue delay compared with other benchmark algorithms.