A Reinforcement Learning Based User Association Algorithm for UAV Networks

There have been increasing interests in employing unmanned aerial vehicles (UAVs) such as drones for telecommunication purpose. In such networks, UAVs act as base stations and provide downloading service to users. Compared with conventional terrestrial base stations, such UAV-BSs can dynamically adjust their locations to improve network performance. However, there exists two important issues in UAV networks, handoff overhead and UAV deployment. The handoff overhead issue is particularly important for UAVs because UAV BSs are connected to cellular BSs via wireless backhaul links, which are costly in terms of spectrum usage and energy consumption. Hence, it is highly desirable to eliminate any unnecessary handoff to minimise the waste of wireless backhaul. The UAV deployment, on the other hand, introduces a new tool for radio resource management, since BS positions are open for network optimisation. In this paper, a smart user association algorithm, named reinforcement learning handoff (RLH), is devised to reduce redundant handoffs in UAV networks and two methods of UAV mobility control are designed to co-operate with the proposed RLH algorithm to optimise the system throughput. In the RLH algorithm, users perform handoffs according to the reward of a reinforcement learning process. In UAV deployment two UAV mobility control methods are proposed respectively base on the SNR estimation and based on the K-Means approach. According to our simulation results, the RLH algorithm can reduce the number of handoffs by 75%.