User Grouping-Based Hierarchical Rate-Splitting Multiple Access for Reconfigurable Intelligent Surface-Assisted Downlink Communication

The joint optimization of rate-splitting multiple access (RSMA) and reconfigurable intelligent surface (RIS) offers a mutually beneficial solution. In this paper, we study a RIS-assisted downlink user grouping-based hierarchical RSMA communication network, where the base station transmits the common streams and private streams simultaneously to serve multiple users. The sum rate (SR) optimization problem is constructed to jointly optimize user grouping, beamforming, and rate splitting. To tackle the NP-hard problem, a user grouping algorithm is first provided to form the hierarchical RSMA model. Next, an equivalent weighted minimum mean-square error (WMMSE) problem is constructed and a WMMSE-based block coordinate descent (BCD) algorithm is designed to update the remaining variables in an iterated way. Specifically, the optimization of transmit beamforming and the rate splitting matrix is reduced to solving a typical convex quadratically constrained quadratic program problem. The solution for passive beamforming is recovered from the intermediate result obtained by the manifold optimization method. Finally, the process of the BCD algorithm is presented. The numerical results indicate that the proposed scheme evidently surpasses one-layer RSMA scheme, NOMA scheme, SDMA scheme, and no RIS scheme, especially in overloaded scenarios. The proposed scheme further increases the SR of common steams compared to the hierarchical RSMA scheme and significantly reduces computational complexity compared to the generalized RSMA scheme.