Adaptive Deployment of UAV-Aided Networks Based on Hybrid Deep Reinforcement Learning

Unmanned aerial vehicles (UAVs) can be used as air base stations to provide fast wireless connections for ground users. Due to their constraints on both mobility and energy consumption, a key problem is how to deploy UAVs adaptively in a geographic area with changing traffic demand of mobile users, while meeting the aforemetioned constraints. In this paper, we propose an adaptive deployment strategy for UAV-aided networks based on hybrid deep reinforcement learning, where a UAV can adjust its movement direction and distance to serve users who move randomly in the target area. Through hybrid deep reinforcement learning, UAVs can be trained offline to obtain the global state information and learn a completely distributed control strategy, with which each UAV only needs to take actions based on its observed state in the real deployment to be fully adaptive. Moreover, in order to improve the speed and effect of learning, we improve hybrid reinforcement learning, by adding genetic algorithms and TD-error-based resampling optimization mechanism. Simulation results show that the hybrid deep reinforcement learning algorithm has better efficiency and robustness in multi-UAV control, and has better performance in terms of coverage, energy consumption and average throughput, by which average throughput can be increased by 20% to 60%.