MMSE-SIC Transceiver Design in Amplify-and-Forward MIMO Relay Systems

Most precoding schemes in amplify-and-forward (AF) multiple-input-multiple-output (MIMO) relay systems only consider the relay precoder, and the receiver is often linear. In this paper, we propose a joint source/relay precoders design method for a nonlinear minimum-mean-squared-error successive interference cancellation (MMSE-SIC) receiver. It is shown that a direct minimization of the average bit-error-rate (BER) is not feasible. Using the uniform channel decomposition (UCD) method, we can design the source and relay precoders such that the signal-to-interference-plus-noise ratio (SINR) for each layer is equal and the block BER (BLER) can be minimized. Unlike UCD in conventional MIMO systems, two precoders have to be optimized simultaneously and the optimum solution is still difficult to derive. We then propose to adopt the primal decomposition method decomposing the original problem into a master and a subproblem optimizations. To facilitate the optimization, we propose a unitary source precoder such that the subproblem can be analytically solved, and the source precoder can be expressed as a function of the relay precoder. Then, with a proposed relay precoder structure, the master problem can be translated into a standard concave optimization problem. By Karush-Kuhn-Tucker (KKT) conditions, we can finally obtain the closed-form solutions for the relay and source precoders. Simulations show that the proposed design can outperform existing precoding methods in AF MIMO relay systems.