Optimizing UAV Location for Deployment in Cell-Free Massive MIMO Networks Using a Soft Actor-Critic Reinforcement Learning

The uncrewed aerial vehicles (UAVs) communication network has recently garnered significant attention and motivation in academic and industrial research due to its flexible mobility concept in providing dynamic coverage in hotspot or congested areas with frequent user demand. While UAVs offer reliable connectivity in such challenging scenarios, integrating them with Cell-Free Massive MIMO (CF-mMIMO) networks presents severe issues in optimal deployment and coverage efficiency. In addressing these issues, we propose a deployment technique using Soft Actor-Critic (SAC), a deep reinforcement learning method, to autonomously determine the optimal placement of the UAV within a given pre-defined coverage area, thereby maximizing channel capacity and throughput. UAVs are tactically deployed to offload traffic from overloaded locations and provide robust communication links with CF-mMIMO access points. We employ a hybrid user association mechanism that combines gradient descent and an enhanced K-means algorithm for efficient user-to-UAV cluster assignment. Simulation results indicate that the SAC-based deployment considerably outperforms traditional static K-means gradient descent-based deployments, achieving a throughput gain of 152.93 Mbps and overall channel capacity gain of 15.709 Gbps. These findings verify the ability of the SAC algorithm to dynamically adjust UAV positions for enhanced performance and smooth connectivity within CF-mMIMO networks, particularly in urban and high-density scenarios.