Joint Transmitter-Receiver Optimization and Self-Interference Suppression in Full-Duplex MIMO Systems

In this paper, we study the effects of joint transmit and receive antennas’ selection on full-duplex (FD) multiple-input-multiple-output (MIMO) networks’ performance. The antennas’ selection problem is, in general, a combinatorial problem whose complexity grows exponentially with the number of antennas. To fully understand the effects of antennas’ selection, we study the sum rate maximization problem in a single-cell network with an FD-MIMO base-station (BS). First, we consider a system with a normal-scale MIMO full-duplex BS, i.e., a normal-scale FD MIMO system. The sum rate maximization problem is studied for two different scenarios; in the first scenario, we consider jointly optimizing the transmit and receive antennas’ selection with the precoder and the receiver weights. A Generalized Bender's Decomposition based algorithm is proposed to solve the mixed-integer nonlinear sum rate maximization problem. In the second scenario, we consider self-interference cancellation via zero-forcing (ZF) transmission. A heuristic algorithm is proposed to solve the sum rate maximization problem by optimizing the selection of the transmit antennas, receive antennas, and the receive antennas at which self-interference is nulled. Second, in a very-large scale, i.e., massive MIMO system, we derive lower bounds for the uplink and downlink rates with ZF receiver and precoder, respectively. The sum rate is maximized by jointly optimizing the transmit to receive antennas ratio and the ratio of the receive antennas at which self-interference is nulled. Finally, via numerical analysis, we evaluate the performance of the formulated sum rate maximization problems.