Optimization and DRL-Based Joint Beamforming Design for Active-RIS Enabled Cognitive Multicast Systems

In this paper, we investigate a cognitive multicast communication system aided by active reconfigurable intelligent surface (active RIS). Specifically, for an underlay spectrum sharing cognitive multicast network, a cognitive radio base station (CRBS) communicates with secondary users (SUs) assisted by an active RIS. Meanwhile, the interference to primary users (PUs) is suppressed within the constraints of the transmit power of both the CRBS and active RIS, together with the restriction of the active RIS amplitude gain. We aim at the fairness problem for maximizing the minimum signal-to-interference-plus-noise-ratio (SINR) via joint beamforming design at the CRBS and the active RIS. To cope with this problem, the optimization and deep reinforcement learning (DRL) based algorithms are proposed. Specifically, the decision variables are decoupled by the alternating optimization (AO) method and then, the non-convex problem is transformed into a solvable convex form by using the successive convex approximation (SCA), Schur complement, and penalty convex-concave procedure (PCCP) methods. Furthermore, we design an AO-based algorithm for the formulated problem. Due to the characteristics of both exploration and exploitation, the DRL-based algorithms outperform the AO-based algorithm with proper parameter settings. Meanwhile, the DRL algorithm inherits the advantages of low execution complexity. The original optimization problem is first converted into a Markov decision process (MDP) form in DRL. Due to the complex objective function and various restrictions of power/amplification gain budget and quality of service (QoS), the constraints are categorized as the switching constraints for action adjustment and performance constraints for reward function setting, respectively. Subsequently, a segmented incentive-based reward function is developed to attain higher performance on SINR. We also propose two effective deep deterministic policy gradient (DDPG)-based and twin delayed deep deterministic policy gradient (TD3)-based algorithms. Finally, the simulation results demonstrate a notable enhancement in system performance upon the introduction of active RIS compared to the case with a passive RIS and the case without using an RIS. Moreover, with appropriately configured parameters, DRL algorithms outperform the AO-based algorithm, and notably, the TD3 algorithm is superior to the DDPG algorithm in optimization effectiveness.