Autonomous Non-Terrestrial Base Station Deployment for Non-Terrestrial Networks: A Reinforcement Learning Approach

Non-Terrestrial Networks (NTN) consisting of multiple space-borne and airborne non-terrestrial base stations (NT-BSs) have recently been introduced by 3GPP as a new paradigm of infrastructure to extend the capacity and coverage of existing terrestrial networks to further support non-terrestrial UEs (NT-UEs). Mobility of NT-BSs and NT-UEs however leads to a dynamic and non-stationary environment, which creates unique challenges in the coverage optimization particularly for the dynamic deployment of multiple NT-BSs. Under the dynamic and non-stationary environment, each NT-BS should autonomously predict not only moving trajectories of NT-UEs and other NT-BSs but also the probability of presence of NT-UEs at any given location, and consequently a new machine learning (ML) scheme is desired. In this paper, instead of adopting the recent innovation of deep reinforcement learning (DRL) approaches inducing an unaffordable complexity for NT-BSs with a limited computing capability, new reinforcement learning (RL) schemes are therefore proposed, by which each of multiple NT-BSs autonomously determines the deployment trajectories to maximize the number of NT-UEs that can access NT-BSs. Through the comprehensive analysis, we justify the convergence of the performance of the proposed schemes. The simulation results also show the effectiveness of the proposed schemes and the outstanding performances as compared with state-of-the-art schemes.