Beamforming Optimization for Generalized MIMO Y Channels with Both Multiplexing and Diversity

We consider a K-user multiple-input multiple-output (MIMO) relay channel, where each user sends independent messages to the other K-1 users via a common relay in two time slots. All users and the relay are equipped with multiple antennas. In contrast to existing work, we consider systems with both multiplexing and diversity, where each user message contains multiple data streams and there are extra degrees of freedom to optimize the transmit beamforming matrices. We propose a novel iterative beamforming optimization algorithm based on orthogonal projection optimization with the signal subspace alignment. An optimal power allocation is also considered to maximize the system sum rate. The sum rate performance of the proposed scheme in various channel configurations is verified by simulations, which shows that the proposed scheme produces significant improvement over existing one.