A Hybrid Scheme Using TOPSIS and Q-Learning for Handover Decision Making in UAV Assisted Heterogeneous Network

An increasing number of users are expected to be served by wireless network with heterogeneous requirements. Unmanned aerial vehicles (UAV) can be deployed to augment the heterogeneous network from aerial area by taking advantages of the characteristics of UAV such as mobility, manoeuvrability, low cost, and Line-of-Sight (LoS) communication. However, the deployment of UAV can also cause problems. For example, in UAV-assisted HetNet, the number of handover (HO) will increase because of the dense distribution of small base stations (SBS) and UAV base stations (UBS). Also, because of the small coverage of SBS and the LoS communication links from neighbour UBSs, the number of unnecessary HO will also rise. Frequent HO and unnecessary HO can result in interruption, increased overhead and energy consumption, which is not desirable for battery powered UAVs. In this paper, to solve the problem, a HO decision making algorithm adopting TOPSIS and Q-learning (QL) is proposed with the aim to reduce HO number and improve energy efficiency. Q-learning can be applied to address decision-making challenges in communication systems widely. However, a large volume of training data can pose challenges and complexities, therefore, the TOPSIS is utilised to reduce the size of the action space in Q-learning. The proposed hybrid TOPSIS-Q-learning method enhances both the handover performance and the scalability. In the method, signal to interference and noise ratio (SINR), time of stay (ToS) and average energy efficiency (EE) are taken into account. The simulation results show that the number of HO and unnecessary HO is remarkably reduced and the average EE is notably improved in comparison with other existing methods.