Deep Reinforcement Learning Aided Rate-Splitting for Interference Channels

Rate-splitting multiple access (RSMA) has emerged as a powerful transmission scheme to mitigate interference in next generation communication systems. In this paper, we investigate precoding with RSMA for a multiple antenna interference channel based on deep reinforcement learning. Specifically, each transmitter has to effectively optimize precoders and allocate transmit power to common and private streams via multiple decision-makers with high dimensional continuous action space. In order to solve this problem, we employ a multi-agent deep deterministic policy gradient (MA-DDPG) framework. In this framework, decentralized agents have partial observability and learn a centralized critic with a multi-dimensional continuous policy. Simulation results show that the proposed rate-splitting method achieves the information theoretical sum-rate upper bound for the single antenna case. It also has a performance very close to the upper bound for the multiple antenna case and outperforms MA-DDPG with no rate-splitting, maximal ratio transmission, zero-forcing and leakage-based precoding methods in both cases.