Sum-Rate Maximization in STAR-RIS-Assisted RSMA Networks: A PPO-Based Algorithm

This article investigates simultaneous transmitting and reflecting reconfigurable intelligent surface (STAR-RIS)-assisted downlink multiuser multiple-input–single-output (MU-MISO) networks with the rate splitting multiple access (RSMA) scheme. A base station (BS) desires to simultaneously transmit messages to multiple users with the assistance of an STAR-RIS to enhance communication quality as well as extend the coverage of users. An optimization problem is formulated to maximize the achievable sum rate of the networks on the premise of satisfying the constraints on power budget at the BS, total common-stream rate of users, and individual users’ minimum rate requirements, via jointly optimizing the beamforming vectors, the common-stream rate allocation vector, and the transmission and reflection coefficients (TARCs) matrix. Due to the dynamic changes of communication links and the coupling of multiple variables, it is challenging to solve such a nonconvex optimization problem by utilizing traditional methods. Therefore, a proximal policy optimization (PPO)-based deep reinforcement learning (DRL) algorithm is proposed, where the reward function, the action space and the state space are designed properly. A constraint-satisfaction-processing (CSP) method is employed to further adjust the optimized transmit power to make sure that the obtained optimized results satisfy the power budget constraint. Simulation results show that the proposed PPO-based DRL algorithm converges well and achieves much better performance than several baselines, such as the soft actor–critic (SAC), the deep deterministic policy gradient (DDPG), the genetic algorithm (GA), the maximum ratio transmission (MRT), the zero-forcing (ZF), and the random methods. Moreover, it demonstrates that deploying STAR-RIS greatly enhances the system sum rate and user fairness compared to deploying traditional reflecting-only RIS (RO-RIS) and without RIS. Besides, it also shows that adopting the RSMA scheme achieves more notable performance gains than the nonorthogonal multiple access (NOMA) scheme in such a network.