Optimal Trajectory and Positioning of UAVs for Small Cell HetNets: Geometrical Analysis and Reinforcement Learning Approach

In this paper, a dynamic unmanned aerial vehicle (UAV)-based heterogeneous network (HetNet) equipped with directional terahertz (THz) antennas is studied to solve the problem of transferring massive traffic of distributed small cells to the core network. To this end, we first characterize a detailed three-dimensional (3D) modeling of the dynamic UAV-assisted HetNet, by taking into account the random positions of small cell base stations (SBSs), spatial angles between THz links, real antenna pattern, and UAV’s vibrations in the 3D space. We then formulate the problem for UAV trajectory to minimize the maximum outage probability (OP) of directional THz links. Then, using geometrical analysis and deep reinforcement learning (RL) method, we propose several algorithms to find the optimal trajectory and select an optimal pattern during the trajectory. For a network with slow time changes, we also propose a deep RL framework to solve the joint optimal UAV positioning and antenna pattern control. The simulation results confirm that the UAV trajectory or antenna pattern control is not enough to achieve acceptable performance, and the UAV should control its antenna patterns during the trajectory to manage the interference.