Hybrid beamforming algorithm using reinforcement learning for millimeter wave wireless systems

In this paper, a Reinforcement Learning (RL) algorithm is presented to speed up the selection process of spatial beams to maximize the mean data rate of a multi-antenna wireless system that implements hybrid beamforming in Millimeter Wave (mmWave) frequency bands. In the proposed hybrid beamforming architecture, the analog beamforming layer is codebook-based, and is implemented using a simple array of phase-shifters that delay the RF signal in the different transmit antennas using a fixed number of discrete steps. In contrast, the digital beamforming layer is much more flexible, and implements a fully adaptive (i.e., non-quantized) digital precoding scheme that enables the simultaneous transmission of few independent base-band data streams in the spatial domain. Obtained simulation results show that the use of RL-based techniques reduces the iterations that are needed to find the most convenient analog beamformers and digital precoders to be used in transmission, without affecting notably the upper bound data rate that is achieved when brute-force search is utilized.