Two-Timescale Hybrid RF-Baseband Precoding With MMSE-VP for Multi-User Massive MIMO Broadcast Channels

This paper explores joint design of two-timescale hybrid RF-baseband precoding with minimum-mean-square-error (MMSE)-vector perturbation (VP) for multi-user massive multiple-input multiple-output systems, where users on the downlink are separated into geographical clusters, and each user cluster experiences identical transmit spatial correlation. Considering the perfect effective channel state information-based MMSE-VP at baseband, the spatial correlation-based RF precoder design is formulated as orthonormality-constrained stochastic optimization problems, where the objective functions cannot be characterized in closed form. RF eigen-beamforming is shown as an optimal solution for single-cluster transmission. In multi-cluster scenarios, mathematically tractable lower bounds are proposed and numerically optimized by trust-region Newton methods on Riemannian manifolds. Additionally, constant-modulus RF precoding based on the discrete Fourier transform (DFT) codebook is addressed. By recognizing the objective functions as a difference of increasing functions, branch-reduce-and-bound techniques are developed to find the globally optimal solutions to such combinatorial problems with reduced computational complexity. Simulation results demonstrate that the proposed nonlinear hybrid schemes deliver a superior bit error rate to other state-of-the-art baselines. The effectiveness of the suboptimal DFT-based RF solutions is also verified.