DNN based Two-stage Compensation Algorithm for THz Hybrid Beamforming with Imperfect Hardware

Terahertz (THz) communication is envisioned as a key technology for 6G and beyond wireless systems owing to its multi-GHz bandwidth. To maintain the same aperture area and the same link budget as the lower frequencies, ultra-massive multi-input and multi-output (UM-MIMO) with hybrid beamforming is promising. Nevertheless, the hardware imperfections particularly at THz frequencies, can degrade spectral efficiency and lead to a high symbol error rate (SER), which is often overlooked yet imperative to address in practical THz communication systems. In this paper, the hybrid beamforming is investigated for THz UM-MIMO systems accounting for comprehensive hardware imperfections, including DAC and ADC quantization errors, in-phase and quadrature imbalance (IQ imbalance), phase noise, amplitude and phase error of imperfect phase shifters and power amplifier (PA) nonlinearity. Then, a two-stage hardware imperfection compensation algorithm is proposed. In the first stage, a deep neural network (DNN) based unified hardware imperfection model is developed to represent the combined hardware imperfections. Furthermore, to balance the performance and model complexity, a tailored network slimming framework is proposed using three slimming methods including pruning, parameter sharing, and power-aware scheme to slim the network in the first stage. In the second stage, the digital precoder in the transmitter (Tx) or the combiner in the receiver (Rx) is designed using neural network (NN) to effectively compensate for these imperfections. Numerical results show that the Tx compensation can perform better than the Rx compensation. Additionally, using the combined slimming methods can reduce parameters by 97.2% and running time by 39.2% while maintaining nearly the same performance in both uncoded and coded systems.