Deep Reinforcement Learning-based Sum-Rate Maximization in Hybrid Beamforming Multi-User Massive MIMO Systems

This work develops deep reinforcement learning (RL)-based techniques to perform power allocation in hybrid beamforming (HB) multi-user massive multiple-input multiple-output (MU-mMIMO) systems. We aim to maximize the achievable system sum-rate constrained to a total transmit power limit. The optimization problem is non-convex and solved in three main steps. First, a radio frequency (RF) beamformer is designed based on the slow time-varying angle of departure (AoD) information of users. Second, a baseband (BB) precoder is derived based on regularized zero-forcing technique using the lower-dimensional effective channel state information (CSI) seen from the BB stage. Afterward, the power allocation (PA) task is formulated in the reinforcement learning framework and it is solved by two main approaches: (i) single-agent deep Q-learning (SA-DQL), (ii) multi-agent deep Q-learning (MA-DQL). An RL agent learns to choose an action (i.e., PA block/value) that maximizes its expected cumulative reward (i.e., sum-rate) through interactions with the environment. It is shown that SA-DQL-based PA can closely approach the performance of exhaustive search in terms of achieved sum-rate and performs significantly better than the MA-DQL-based PA and equal PA scenarios.