Deep Reinforcement Learning for Green UAV-Assisted Data Collection

Due to high maneuverability and flexible deployment, unmanned aerial vehicles (UAVs) are emerging as an alternative for reliable wireless communications. The main challenge of integrating UAVs with cellular networks is their limited on-board energy capacity, which restricts their operation period. Hence, this article examines the energy-efficiency (EE) maximization under the constraint of UAV’s propulsion and data reception energy. Specifically, the formulated problem optimizes the user associations with UAV or base station, their respective transmit power allocations, and UAV’s trajectory subject to the user data rate requirements. As this joint optimization problem is combinatorial and involves multiple variables, we have reduced it into an equivalent tractable form using the Markov decision process (MDP). Later we leverage the deep reinforcement learning (DRL) framework based on a deep deterministic policy gradient (DDPG) algorithm to learn the UAV’s trajectory. The proposed green DRL algorithm improves the total EE of the system by 15.63% compared to the benchmark particle swarm optimization.