Access Point Clustering in Cell-Free Massive MIMO Using Multi-Agent Reinforcement Learning

Conventional massive multiple-input multiple-output (MIMO) systems provide high spectral efficiency, throughput, and energy efficiency, but suffer from high inter-cell interference and poor cell-edge coverage. Cell-free massive MIMO addresses these shortcomings by geographically distributing access points (APs), each with one or more antennas, that form a virtual massive MIMO array instead of co-locating all antennas at a base station. This distributed AP placement significantly reduces the average distance between a user equipment (UE) and an AP. Existing work mainly considers the ideal canonical case where each UE is served by all APs, which is impractical because of limited fronthaul capacity and finite computational resources. Therefore, to make the network scalable to an arbitrary size, a user-centric approach should be adopted, where each UE is served by a personalized cluster of nearby APs. However, the clustering problem is combinatorially complex and may be too time consuming to solve optimally when the UEs are in motion. Therefore, in this work, we develop a multi-agent reinforcement learning (MARL) algorithm for AP selection and clustering, where each AP is an agent in the MARL algorithm and trained to near-optimally select for itself which UEs to serve. Simulation results demonstrate our MARL algorithm outperforms typical AP selection algorithms such as greedy selection as well as more sophisticated ones from the literature, and also provides performance comparable to the ideal canonical case.