Hybrid Beamforming in mm-Wave MIMO Systems Having a Finite Input Alphabet

Recently, there has been significant research effort toward achieving high data rates in the millimeter wave bands by employing large antenna systems. These systems are considered to have only a fraction of the RF chains compared with the total number of antennas and employ analog phase shifters to steer the transmit and receive beams in addition to the conventional beamforming (BF)/combining invoked in the baseband domain. This scheme, which is popularly known as hybrid BF, has been extensively studied in the literature. To the best of our knowledge, all the existing schemes focus on obtaining the BF/combining matrices that maximize the system capacity computed using a Gaussian input alphabet. However, this choice of matrices may be suboptimal for practical systems, since they employ a finite input alphabet, such as quadrature amplitude modulation/phase-shift keying constellations. Hence, in this paper, we consider a hybrid BF/combining system operating with a finite input alphabet and optimize the analog as well as digital BF/combining matrices by maximizing the mutual information (MI). This is achieved by an iterative gradient ascent algorithm that exploits the relationship between the minimum mean-squared error and the MI. Furthermore, an iterative algorithm is proposed for designing a codebook for the analog and digital BF/combining matrices based on a vector quantization approach. Our simulation results demonstrate that the proposed gradient ascent algorithm achieves an ergodic rate improvement of up to 0.4 bits per channel use (bpcu) compared with the Gaussian input scenario. Furthermore, the gain in the ergodic rate achieved by employing the vector quantization-based codebook is about 0.5 bpcu compared with the Gaussian input scenario.