Information Timeliness Driven Statistical QoS Guarantee in RIS-Enabled Wireless Networks via Deep Reinforcement Learning

The randomness and complexity of the wireless channel is challenging to meet the various quality of service (QoS) for different wireless communication application scenarios. Reconfigurable intelligent surface (RIS) technology has been proposed to achieve dynamic control of signal propagation over the wireless medium, and thus enables intelligent reconstruction of the channel environment. Moreover, age of information (AoI) has been proposed to quantify the timeliness of status update information accurately, which is new QoS metric. However, the AoI-driven statistical QoS guarantee problem in the RIS-enabled wireless network is not trivial and needs to be solved. In this paper, we employ the AoI violation probability to measure the reliability requirement for maintaining the freshness of status updates and derive its upper bound. Then, we formulate the AoI-driven effective capacity maximization problem. Finally, we transform the formulated problem into a signal-to-noise ratio (SNR) maximization problem, and further propose a twin delayed deep deterministic policy gradient (TD-DDPG) based joint optimization algorithm for obtaining the effective decisions on transmission power of device and the phase shift of RIS. Simulation results show that the TD-DDPG-based scheme has better performance than other traditional schemes.