Optimization of Broadcast Beams in Massive MIMO: Learning from A Digital Twin

Broadcast beam design is essential for enhancing the network coverage and improving the quality of service (QoS) in massive multiple-input-multiple-output (MIMO) communications. To improve the performance in dynamic scenarios with varying user numbers and distributions, we leverage deep reinforcement learning (DRL) to optimize the broadcast beam and propose a multi-agent DRL (MA-DRL) framework to handle the extremely high action dimension. To reduce the performance fluctuations in DRL, we construct a data-driven digital twin (DT) as a virtual environment for the initial training phase, while using an assistant baseline scheme to ensure acceptable real-time performance. Numerical results demonstrate that the proposed DT-enabled MA-DRL approach not only enhances the coverage, but also enables safe exploration and rapid adaptation to varying user numbers and distributions. Meanwhile, a faster convergence speed is provided since considerable communication overhead required in the real network can be saved in the DT.