Deep Reinforcement Learning Algorithms for Location Optimization in Multi-RAT UAV-Assisted Heterogeneous Networks

Due to the unprecedented growth in wireless mobile connectivity, the next generations of wireless networks are required to meet the different requirements of the diverse devices. One of the promising solutions for the wireless network design is the Unmanned Aerial Vehicles (UAVs), which provide flexibility and dynamic mobility, that can be exploited to extend capacity for unexpected and unplanned increases in demand. In this paper, different Deep Reinforcement Learning (DRL) approaches along with different reward functions are compared, to optimize the 3D location of an Unmanned Aerial Vehicle (UAV) carrying Multiple Radio AccessTechnologies (Multi-RAT) base stations in a Multi-RAT Heterogeneous Network (HetNet) environment, based on the network conditions and users' demands. The idea of deploying Multi-RAT base stations on a UAV will increase the system capacity by exploiting the same resources, however, it will be more challenging to optimize the complex UAV's 3D location to maximize the users' experience. Extensive simulations are conducted to evaluate and compare the performance of the different DRL approaches, and the different reward functions, which revealed that the instantaneous reward function and the Double Deep Q-Network (DDQN) algorithm are more effective in terms of average user satisfaction and stability in solving the proposed optimization problem.