Compensation of Transmitter IQ Imbalance in Multi-User Hybrid Beamforming Systems

In this paper, we consider the uplink (UL) communication of a massive multi-user (MU) multiple-input multiple-output (MIMO) hybrid beamforming system (HBFS) in which the transmitting user-equipments are affected by IQ imbalance (IQI). We first show that if the transmitter IQI is not compensated, it will result in a finite ceiling of the UL achievable sum-rate at high signal-to-noise ratio, which only depends on the transmitter IQI matrices and is independent of the propagation channel and the choice of the hybrid combining matrices. This justifies the need for transmitter IQI compensation in massive MU-MIMO HBFS. Therefore, we propose a novel zero-forcing based transmitter IQI compensation algorithm to be implemented at the base station which effectively mitigates the undesired effects of transmitter IQI and is applicable for any channel model and any choice of the number of RF chains. For uncorrelated Rayleigh fading channel, we derive an approximate closed-form expression of the UL sum-rate achieved by the massive MU-MIMO HBFS with the proposed transmitter IQI compensation. Finally, numerical results are presented which confirm the effectiveness of the proposed transmitter IQI compensation algorithm in mitigating the undesired effects of transmitter IQI in different channel environments.