Autoencoder Communications with Optimized Interference Suppression for NextG RAN

This paper models an end-to-end communications system for the NextG radio access network (RAN) as an autoencoder (AE) subject to interference effects. The transmitter (coding and modulation) and receiver (demodulation and decoding) are represented as deep neural networks (DNNs) of the encoder and decoder, respectively. The AE communications systems is trained with interference training and randomized smoothing to operate under unknown and dynamic interference (jamming) effects. Compared to conventional communications, the AE communications with interference training and randomized smoothing can achieve up to 36 dB interference suppression with a channel reuse of four for the single antenna case. This paper also extends the AE communications formulation to the multiple-input multiple-output (MIMO) case under interference effects and shows the bit error rate (BER) improvement compared to conventional MIMO communications.