Dynamic Antenna Configuration for 3D Massive MIMO System via Deep Reinforcement Learning

We study the optimized dynamic antenna parameters configuration for the 3D massive multiple-input multiple- out (MIMO) system in a heterogeneous network (HetNet) with overlaid macrocells and smallcells. In particular, we propose a deep reinforcement learning (DRL) approach to jointly adjust three key antenna parameters, namely, downtilt angle, vertical and horizontal half-power beamwidths of the macro base stations (mBSs) automatically in a dynamic environment with strong user mobility. More specifically, employing the gridded user location information (ULI), we propose a novel mix Q-learning algorithm to efficiently address the challenging joint optimization problem, which integrates a parallel hyper-parameter updating mechanism in dual sub-networks and a technique of prioritized replay buffer. The resultant neural network can efficiently learn the historical experience in an online fashion and achieve excellent sum-rate performance with affordable trials. Moreover, thanks to the proposed gridded ULI, our DRL-empowered antenna configuration framework can easily fit various HetNet deployments with variable user densities. Numerical results show that the average weighted sum-rate is increased by 4.59 bit/s/Hz, and the average performance improvement is up to 24.82% as compared to the reference scheme without gridded ULI.