Robust Beamforming and Phase Shift Control in RIS-UAV Relay System Using Deep Reinforcement Learning

Unmanned Aerial Vehicles (UAVs) and other aerial communication devices are crucial components of the sixth-generation (6 G) network, serving as airborne mobile access points to provide services to a moving ground user (GU). This paper considers an air-ground communication system where a base station sends signals to the moving GU via a UAV relay. To better cope with complex communication environments, such as urban areas with obstacles, we introduce an onboard Reconfigurable Intelligent Surface (RIS) on the UAV to enhance the communication performance of the system. By jointly optimizing the UAV's flight trajectory, base station beamforming, and IRS phase shifts, we aim to maximize the total achievable rate of the system. This problem is difficult to solve. Instead, we reformulate it as a Markov Decision Process (MDP). A deep reinforcement learning (DRL) algorithm based on a Soft Actor-Critic (SAC) is proposed to solve the problem. Simulation results show that the onboard RIS, compared to a fixed RIS, can effectively improve the total achievable rate of the system. Additionally, our proposed algorithm achieves better performance compared to some benchmark algorithms.