Optimized MIMO transmission and compression for interference mitigation with cooperative relay

This paper considers a novel use of device-to-device link for cooperative communication wherein a nearby user terminal acts as a relay in enabling both signal enhancement and common interference rejection at the intended destination. Assuming Gaussian transmission and Gaussian compress-and-forward relaying strategy for the multiple-input multiple-output (MIMO) relay channel with a finite-capacity out-of-band relay-destination link and with arbitrarily correlated noises, this paper proposes a coordinate ascent approach for iteratively optimizing the transmit covariance matrix at the source and the quantization noise covariance matrix at the relay. We show that the optimization of quantization noise covariance matrix under fixed input can be solved in closed form using a simultaneous diagonalization approach, while the optimization of transmit covariance matrix under fixed quantization can be cast as a convex optimization problem. This paper further introduces the concept of antenna pooling and illustrates the importance of accounting for the noise correlation across the user terminals due to common interference. We show that the optimized transmission and device-to-device relaying strategies that take advantage of the noise correlation can significantly improve the user throughput in a cellular environment by enabling interference rejection across the user terminals.