Channel Estimation and Hybrid Precoding for Millimeter Wave Communications: A Deep Learning-Based Approach

Hybrid analog and digital beamforming (HBF) has been regarded as a key technology for future millimeter wave (mmWave) communication systems due to its ability to obtain a good trade-off between achievable beamforming gain and hardware cost. In this paper, we investigate the channel estimation and hybrid precoding for mmWave MIMO systems with deep learning. We adopt the hierarchical codebook based algorithm for channel estimation as it requires limited number of pilot transmissions, and enhance its performance by proposing a new codebook design algorithm based on manifold optimization (MO). With the estimated channel state information (CSI) as the input, we develop a robust HBF network (HBF-Net) by applying convolutional layers and attention mechanism, which can be trained to generate a robust HBF matrix targeting at spectral efficiency maximization with imperfect CSI. To further improve the performance, we propose a joint channel estimation and HBF optimization network (CE-HBF-Net). Considering that the adaptively selected HBF vectors in the hierarchical codebook based channel estimation are different for different channel realizations, we skillfully propose an index assign-and-input method to efficiently feed such information to the CE-HBF-Net to reduce the network input dimensions and make the network trainable. Furthermore, we propose a signal self-attention mechanism to enable the CE-HBF-Net to intelligently assign larger weight coefficients to those signals that contribute more to channel estimation. Simulation results show that the well-designed HBF-Net and CE-HBF-Net outperform the conventional HBF algorithms with imperfect channel and exhibit robustness to mismatches between offline training and online deployment stages.