Soft Actor–Critic Based 3-D Deployment and Power Allocation in Cell-Free Unmanned Aerial Vehicle Networks

This letter investigates a cell-free unmanned aerial vehicles (UAV) network where UAVs serve as flying access points (FAPs) to overcome the inter-cell interference in conventional cell-based UAV networks and offer more uniform service. Considering the fairness among users, we aim to maximize the minimum user rate by joint optimizing three-dimensional (3D) deployment and power allocation of FAPs. To solve this complex high-dimentional non-convex problem, we firstly formulate it as a constrained Markov Decision Process (CMDP) guaranteeing the safety distance between FAPs. Then a soft actor-critic (SAC) based deep reinforcement learning (DRL) algorithm is proposed which adds maximum entropy term to the objective function. Simulation results demonstrate that the cell-free UAV network can achieve a higher minimum user rate than cell based UAV network, and SAC-based algorithm outperforms conventional DRL-based algorithms in terms of stability and long-term rewards.