Transceiver Design in Millimeter Wave Full-Duplex Multi-User Massive MIMO Communication Systems

We consider hybrid transmit precoding and receive combining in a millimeter wave (mmWave) multiple-input multiple-output (MIMO) multi-user cellular system, where a full-duplex (FD) base station (BS) serves multiple half-duplex (HD) uplink and downlink users simultaneously. Jointly deriving the four matrices of analog precoder, digital precoder, analog combiner, and digital combiner for mmWave FD communications is intractable. Hence, we propose two approaches for either designing joint analog and digital beamformers separately at the transmitters and receivers, or jointly deriving transmit precoders and receive combiners with separate designs of analog and digital beamforming matrices that maximize the sum-rate. In the first proposed method, at the first step, we derive an iterative solution for the fully digital minimum-mean-square-error (MMSE) transmit/receive beamforming. Then, exploiting the spatial structure of mmWave channels, sparse approximation problems are formulated to jointly design radio frequency (RF) and baseband filters approximating the fully digital beamformers via orthogonal-matching-pursuit (OMP) algorithm. To reduce the computational complexity of the first proposed method, which is mainly caused by the iterations between the transmit and receive beamformings, in the second presented scheme, transmit precoding and receive combining matrices are calculated jointly, while RF and digital beamformings are derived separately. More specifically, digital beamforming matrices are derived using the block diagonalization (BD) approach to mitigate the intra-user and inter-users interference, and analog filters are calculated via solving sum-rate maximization problem. In the simulation results, it is shown that our proposed OMP-based hybrid beamforming (HBF) scheme outperforms other HBF approaches for mmWave FD communications in the literature. In addition, it is shown that when the number of RF chains is at least double the number of data streams, the proposed BD-based HBF further improves over the OMP-based HBF in terms of sum-rate. Thus, choosing between OMP-based and BD-based HBF depends on the number of available RF chains.