Deep Reinforcement Learning for Robust RIS-Aided Over-the-Air Federated Learning in Cognitive Radio

With the aim of exploring the integration of federated learning (FL) into wireless networks, particularly in light of the challenge of spectrum resource limitations, this paper considers the integration of over-the-air FL (OTA-FL) and cognitive radio networks (CRN) in the presence of uncertainties in channel gains. Furthermore, this study proposes the utilization of reconfigurable intelligent surface (RIS) to enhance OTA-FLwithin the secondary network (SN) of CRN, leveraging RIS capabilities to mitigate interference to the primary network (PN). Our focus lies in minimizing the mean squared error (MSE) of model aggregation, considering the uncertainty in channel estimation and total interference constraints. We formulate this optimization challenge as a Markov decision process (MDP) and exploit a deep reinforcement learning (DRL)-based approach to effectively address this complex problem. Finally, simulation outcomes validate the outperform of the proposed approach.