Mobility-Aware Trajectory Design for Aerial Base Station Using Deep Reinforcement Learning

In this paper, an unmanned aerial vehicle (UAV) assisted wireless network is investigated, where an aerial base station (ABS) is deployed for serving both ground and aerial users. The objective is to maximize the uplink sum-rate of all users by designing the trajectory of the ABS, while considering the mobility of users. Specifically, it is assumed that all aerial UAVs move individually with the random waypoint mobility model (RWM). Meanwhile, some ground users move with the reference point group mobility model, while others follow the RWM as well. Due to the mobility of both ground and aerial users, conventional reinforcement learning (such as Q-learning) is very likely to fail for the curse of dimensionality. Therefore, it is non-trivial to solve the trajectory design problem in the large-scale highly dynamic environment. By approximating the value functions with neural networks, a deep reinforcement learning-based algorithm is proposed for the ABS to autonomously provide network services in the rapidly changing environment. Simulation results demonstrate that our designed double deep Q-network (DDQN) can learn the environment information very well with our elaborate rewards. Compared to the conventional DDQN with fully connected network, our proposed DDQN converges faster and has better stability.