A Two-Tier Deep Neural Network Detector for Two-User Rate-Splitting Multiple Access Systems

In this paper, we propose a two-tier, deep neural network (DNN)-based detector for a two-user, multi-antenna-equipped rate-splitting multiple access (RSMA) uplink system. The proposed DNN model primarily consists of a long short-term memory architecture, with one network that decodes the common streams and another that decodes the private streams, in an orthogonal frequency division multiplexing (OFDM) setting. The DNN-based detector is designed to decode RSMA messages from the superimposed signal at the base station (BS) in one go, and overcomes the disadvantages of a successive interference cancellation-based receiver. Through Monte Carlo simulations, we show that the proposed detector nearly attains the performance of the optimal maximum likelihood detector for various modulation schemes on common and private streams with OFDM. Further, we study the effect of number of transmit antennas and the DNN hyperparameters on the performance of the proposed detector, in terms of the symbol error rate.