Deep Learning-based Spectral Efficiency Maximization in Massive MIMO-NOMA Systems with STAR-RIS

This paper studies a deep learning-based framework for spectral efficiency maximization problem in massive multiple-input multiple-output (MIMO)-non-orthogonal multiple access (NOMA) systems with simultaneously transmitting and reflecting reconfigurable intelligent surface (STAR-RIS). We formulate the spectral efficiency maximization with a joint design of power allocation of the users, phase shift matrix of transmission and reflection element at the STAR-RIS. Since the problem is non-convex and power allocation of the users and reflector/transmitter elements at a STAR-RIS are coupled, it is very challenging to solve optimally. We propose a low-complexity iterative algorithm based on the inner approximation (IA) method to solve this problem with guaranteed convergence at a relatively optimal level. For real-time optimization, we design a deep learning (DL) framework to predict the optimal solution of power allocation of users, phase shift matrix of transmission and reflection elements at the STAR-RIS according to distances and channel gains from the base station (BS) to STAR-RIS and from STAR-RIS to users. Simulation results show that the suggested scheme improves the spectral efficiency (SE) compared to the massive MIMO system with direct link and without STAR-RIS. Besides, the DL framework can predict the optimal solution within a short time under the suggested scheme.