Secure Beamforming for MIMO Two-Way Communications With an Untrusted Relay

This paper studies the secure beamforming design in a multiple-antenna three-node system where two source nodes exchange messages with the help of an untrusted relay node. The relay acts as both an essential signal forwarder and a potential eavesdropper. Both two-phase and three-phase two-way relay strategies are considered. Our goal is to jointly optimize the source and relay beamformers for maximizing the secrecy sum rate of the two-way communications. We first derive the optimal relay beamformer structures. Then, iterative algorithms are proposed to find source and relay beamformers jointly based on alternating optimization. Furthermore, we conduct asymptotic analysis on the maximum secrecy sum-rate. Our analysis shows that when all transmit powers approach infinity, the two-phase two-way relay scheme achieves the maximum secrecy sum rate if the source beamformers are designed such that the received signals at the relay align in the same direction. This reveals an important advantage of signal alignment technique in against eavesdropping. It is also shown that if the source powers approach zero, the three-phase scheme performs the best while the two-phase scheme is even worse than direct transmission. Simulation results have verified the efficiency of the proposed secure beamforming algorithms as well as the analytical findings.