Deep Neural Hybrid Beamforming for Multi-User mmWave Massive MIMO System

Hybrid beamforming (HB) has emerged as a promising technology to support ultra high transmission capacity and with low complexity for Millimeter Wave (mmWave) multiple-input and multiple-output (MIMO) system. However, the design of digital and analog beamformer is a challenge task with non-convex optimization, especially for the multi-user scenario. Recently, the blooming of deep learning research provides a new vision for the signal processing of communication system. In this work, we propose a deep neural network based HB for the multi-User mmWave massive MIMO system, referred as DNHB. The HB system is formulated as an autoencoder neural network, which is trained in a style of end-to-end self-supervised learning. With the strong representation capability of deep neural network, the proposed DNHB exhibits superior performance than the traditional linear processing methods. According to the simulation results, DNHB outperforms about 2 dB in terms of bit error rate (BER) performance compared with existing methods.