Dynamic Power Allocation for Cell-Free Massive MIMO: Deep Reinforcement Learning Methods

Power allocation plays a central role in cell-free (CF) massive multiple-input multiple-output (MIMO) systems. Many effective methods, e.g., the weighted minimum mean square error (WMMSE) algorithm, have been developed for optimizing the power allocation. Since the state of the channels evolves in time, the power allocation should stay in tune with this state. The present methods to achieve this typically find a near-optimal solution in an iterative manner at the cost of a considerable computational complexity, potentially compromising the timeliness of the power allocation. In this paper we address this problem by exploring the use of data-driven methods since they can achieve near-optimal performance with a low computational complexity. Deep reinforcement learning (DRL) is one such method. We explore two DRL power allocation methods, namely the deep Q-network (DQN) and the deep deterministic policy gradient (DDPG). The objective is to maximize the sum-spectral efficiency (SE) in CF massive MIMO, operating in the microwave domain. The numerical results, obtained for a 3GPP indoor scenario, show that the DRL-based methods have a competitive performance compared with WMMSE and the computational complexity is much lower. We found that the well-trained DRL methods achieved at least a 33% higher sum-SE than the WMMSE algorithm. The execution times of the DRL methods in our simulation platform are 0.1% of the WMMSE algorithm.