Deep Reinforcement Learning Based Dynamic Beam Selection in Dual-Band Communication Systems

To reduce the downlink beam sweep overhead of mmWave systems, we propose a deep reinforcement learning based dynamic beam selection (DRL-DBS) method. A new learning motivation is presented by analyzing the dynamic change laws of high- and low-frequency channels in the spatial domain: to learn the index offset between the optimal beam of mmWave and sub-6 GHz spatial spectrum. In the DRL-DBS method, we propose a novel action space where actions can dynamically adjust the size of the beam sweep subset according to the high- and low-frequency channel propagation laws. Hence, the DRL-DBS method can predict a mmWave downlink beam sweep subset with dynamic size, and the optimal beamforming index is from beam sweep results on the subset. A dual-input dueling Q-network with noisy networks and prioritized experience replay is designed to select the optimal action. The DRL-DBS method can achieve a dynamic trade-off between mmWave beam selection quality and beam sweep overhead based on the reward function. Simulation results demonstrate the superior performance of the DRL-DBS method compared with the existing strategies. Especially, the DRL-DBS method outperforms the exhaustive search algorithm in achievable rate because the overhead of mmWave beam sweep is considered.