Joint flexible duplexing and power allocation with deep reinforcement learning in cell-free massive MIMO system

Network-assisted full duplex (NAFD) cellfree (CF) massive MIMO has drawn increasing attention in 6G evolvement. In this paper, we build an NAFD CF system in which the users and access points (APs) can flexibly select their duplex modes to increase the link spectral efficiency. Then we formulate a joint flexible duplexing and power allocation problem to balance the user fairness and system spectral efficiency. We further transform the problem into a probability optimization to accommodate the short-term communications. In contrast with the instant performance optimization, the probability optimization belongs to a sequential decision making problem, and thus we reformulate it as a Markov Decision Process (MDP). We utilizes deep reinforcement learning (DRL) algorithm to search the solution from a large state-action space, and propose an asynchronous advantage actor-critic (A3C)-based scheme to reduce the chance of converging to the suboptimal policy. Simulation results demonstrate that the A3C-based scheme is superior to the baseline schemes in term of the complexity, accumulated log spectral efficiency, and stability.