Low Complexity Online RL Enabled UAV Trajectory Planning Considering Connectivity and Obstacle Avoidance Constraints

In recent years, we have witnessed a significant proliferation of UAV s to service variety of verticals, including last-mile delivery, medical assistance, crop monitoring etc. The true utility of these UAV s lies in beyond-line-of-sight (BLoS) operations which require wide-area connectivity. The increased densification of the cellular networks under 5G networks is ideally suited for this purpose. However, traditional cellular networks are not optimised for servicing aerial clients. In this paper, we propose a novel approach to improve connectivity for delivery UAV s with the ground base stations using low complexity online reinforcement learning (RL) and multiple Q-learning algorithms. We demonstrate that by training multiple Q-learning models, we can accurately predict optimal trajectory for UAV s flight. The optimality is in the sense of its shortest flight path while ensuring continuity in connectivity with the ground base station. Our modeling explicitly incorporates dynamic changes in the environment which may not be observed during the previous missions by UAV. We implement our proposed methodology using OpenAI Gym and demonstrate that the proposed approach allows perpetual connectivity for UAV s while generating obstacle-free flight trajectories.