Sparse Bayesian Learning Using Complex t-Prior for Beam-Domain Massive MIMO Channel Estimation

This paper proposes a novel beam-domain channel estimation (CE) algorithm via sparse Bayesian learning (SBL) using complex t-prior for massive multi-user multiple-input multiple-output (MIMO) systems. Due to the sidelobe leakage and insufficient observation resolution resulting from physical constraints, the equivalent channel after digital beamforming at the receiver has a structure with many small but non-zero elements, which cannot be modeled strictly as a sparse signal. To fully capture this pseudo-sparse structure characterized by the signal strength variations among elements, we design a novel SBL algorithm that incorporates a complex t-distribution using a hierarchical Bayesian model. By utilizing a high degree of adaptability of this heavy-tailed prior, it is possible to efficiently learn the signal strength, accounting for elements with non-zero but small values, which is verified by the regularization analysis based on an equivalent optimization problem. The efficacy of the proposed CE algorithm is confirmed by numerical simulations, which show that the proposed method not only significantly outperforms the state-of-the-art (SotA) sparse signal recovery (SSR)-based algorithms but also achieves the performance of a genie-aided scheme over a wide signal-to-noise ratio (SNR) range in both sub-6 GHz and millimeter-wave (mmWave) wireless communication scenarios.