Adaptive 3D Placement of Multiple UAV-Mounted Base Stations in 6G Airborne Small Cells With Deep Reinforcement Learning

Uncrewed Aerial Vehicle-mounted Base Stations (UAV-BSs) have been envisioned as a promising solution to enable high-quality services in next-generation mobile networks. With inherent flexibility, one key challenge is placing the UAV-BSs adaptively to time-varying network conditions to maintain stable connections. Conventional methods mainly focus on optimizing UAV-BS deployment in static networks where users are static or with limited mobility. This study considers a dynamic network where multiple non-stationary UAV-BSs are deployed to serve mobile users with time-varying heterogeneous traffic. Incomplete downloads of users are backlogged in download queues at the UAV-BSs, and together with the newly requested traffic, the download queue size reflects the user’s instantaneous traffic demand. With constraints on the queue stability, we aim to maximize the user’s long-term mean opinion score (MOS), reflecting how the perceived data rate satisfies their traffic demand. Since the users’ location and traffic demand vary over time, we dynamically group users into clusters using a K-means-based algorithm and adjust the 3D location of UAV-BSs using an actor-critic deep reinforcement learning (DRL) framework. The actor module is encoded using a deep neural network (DNN) that obtains the UAV-BS’s current location and a traffic heatmap of users to predict the optimal movement for UAV-BSs. The critic module utilizes Lyapunov optimization to control the queue stability constraint and evaluate the actor’s decisions. Extensive simulations demonstrate the proposed method’s superior performance over conventional rate-maximization approaches.