DRL Enabled Coverage and Capacity Optimization in STAR-RIS-Assisted Networks

Simultaneously transmitting and reflecting reconfigurable intelligent surfaces (STAR-RISs) is a promising passive device that contributes to full-space coverage via transmitting and reflecting the incident signal simultaneously. As a new paradigm in wireless communications, how to analyze the coverage and capacity performance of STAR-RISs becomes essential but challenging. To solve the coverage and capacity optimization (CCO) problem in STAR-RIS-assisted networks, a multi-objective proximal policy optimization (MO-PPO) algorithm is proposed to handle long-term effects. To strike a balance between each objective, the MO-PPO algorithm provides a set of optimal solutions to approach a Pareto front (PF), where the solution on the approximate PF is regarded as an optimal result. Moreover, in order to improve the performance of the MO-PPO algorithm, two update strategies, i.e., action-value-based update strategy (AVUS) and loss function-based update strategy (LFUS), are investigated. For the AVUS, the improved point is to integrate the action values of both coverage and capacity and then update the loss function. For the LFUS, the improved point is only to assign dynamic weights for both loss functions of coverage and capacity, while the weights are calculated by a min-norm solver at every update. The numerical results demonstrated that the investigated update strategies outperform the fixed weights MO optimization algorithms in different cases, which include a different number of sample grids, the number of STAR-RISs, the number of elements in the STAR-RISs, and the size of STAR-RISs. Additionally, the STAR-RIS-assisted networks achieve better performance than conventional wireless networks without STAR-RISs. Moreover, with the same bandwidth, a millimetre wave is able to provide higher capacity than sub-6 GHz, but at a cost of smaller coverage.