An Improved Q-Learning Based Handover Scheme in Cellular-Connected UAV Network

Cellular-Connected Unmanned Aerial Vehicles (UAVs) have been used for a variety of applications, from remote sensing, monitoring to the search and rescue operations. Many applications of UAVs rely on the seamless and reliable communication link to control the UAVs remotely and transmit the data. Thus, the challenges due to the mobility in three dimensional space, such as the air-to-ground channels, interference and handover (HO) problems must be addressed. The HO failure and unnecessary HO seriously affect the quality of service. Q-learning is an effective method to address the challenges in HO and has attracted a lot of attention. In this work, we introduce some changes in an existing HO algorithm and proposed an improved Q-Learning based HO algorithm. The formation of the Action Space considers both the Signal-to-Interference-plus-noise ratio (SINR) and the distance between Base station (BS) and UAV with different weights. The results show the proposed algorithm can further reduce the HO rate by increasing the weight of distance with slightly compromising the throughput and the outage rate. An optimal distance weight is suggested based on the analysis of the three performance indicators.