Optimal Design for Trajectory and Phase-Shift in UAV-Mounted-RIS Communications with Reinforcement Learning

Taking advantages of both unmanned aerial vehicles (UAVs) and reconfigurable intelligent surfaces (RISs), UAV-mounted-RIS systems are expected to enhance transmission performance in complex wireless environments. In this paper, we focus on system design for a UAV-mounted-RIS system and investigate joint optimization for the RIS’s phase shift and the UAV’s trajectory. To cope with the practical issue of inaccessible information on the user terminals’ (UTs) location and channel state, a reinforcement learning (RL)-based solution is proposed to find the optimal policy with finite steps of "trial-and-error". However, the online interaction between the agent and environment may cause instability during the training and the assumption of (first-order) Markovian state transition may be impractical in real-world problems. Therefore, the decision transformer (DT) algorithm is employed for RL model training to adapt to more general state transition. Experimental results demonstrate that the proposed RL solutions are efficient in model training and present acceptable performance close to that achieved by the benchmark, which relies on conventional optimization algorithms with the UT’s locations and channel parameters explicitly known beforehand.