Deep reinforcement learning based joint cooperation clustering and downlink power control for cell-free massive MIMO

In recent times, various power control and clustering approaches have been proposed to enhance overall performance for cell-free massive multiple-input multiple-output (CF-mMIMO) networks. With the emergence of deep reinforcement learning (DRL), significant progress has been made in the field of network optimization as DRL holds great promise for improving network performance and efficiency. In this work, our focus delves into the intricate challenge of joint cooperation clustering and downlink power control within CF-mMIMO networks. Leveraging the potent deep deterministic policy gradient (DDPG) algorithm, our objective is to maximize the proportional fairness (PF) for user rates, thereby aiming to achieve optimal network performance and resource utilization. Moreover, we harness the concept of “divide and conquer” strategy, introducing two innovative methods termed alternating DDPG (A-DDPG) and hierarchical DDPG (H-DDPG). These approaches aim to decompose the intricate joint optimization problem into more manageable sub-problems, thereby facilitating a more efficient resolution process. Our findings unequivocally showcase the superior efficacy of our proposed DDPG approach over the baseline schemes in both clustering and downlink power control. Furthermore, the A-DDPG and H-DDPG obtain higher performance gain than DDPG with lower computational complexity.