Joint Tx–Rx Beamforming for Optimal Gain and Self-Interference Mitigation in In-Band Full-Duplex Arrays: Theory, Figures of Merit, and Validation

We introduce a novel joint transmit (Tx) and receive (Rx) beamforming method and relevant performance metrics for in-band full-duplex (IBFD) antenna systems, addressing two major challenges: self-interference (SI) leakage from the transmitter to the receiver and nonlinear gain compression in the receiver's low-noise amplifiers (LNAs). Conventional methods treat Tx and Rx beamforming separately, often failing to balance SI suppression with sufficient Tx gain. In contrast, our approach achieves two key objectives. First, it maximizes Tx array antenna gain while ensuring that SI at the LNA input remains below a predefined threshold, preventing saturation and maintaining linearity. Second, it suppresses residual SI at the receiver output, improving the signal-to-SI ratio by effectively separating the desired signal from residual interference. This method is validated using a commercial RF system-on-chip evaluation board and 3 × 5 Tx and Rx Vivaldi antenna arrays operating in the 3 GHz to 6 GHz band. Experimental results show that the conventional Tx maxGain beamformer achieves approximately 22 dB isolation between closely spaced Tx and Rx arrays. In contrast, the proposed Tx beamforming method provides over 45 dB isolation, with a Tx array gain of up to 10 dBi. Overall, the joint Tx–Rx beamforming method achieves more than 80 dB isolation with a 10 dBi Rx array gain for the IBFD system, representing a significant improvement over conventional methods.