Accelerated Constrained Reinforcement Learning Based Energy-Efficient Power Scheduling Algorithm for Extended Reality Transmission

In the emerging extended reality (XR) applications, downlink transmission struggles with tricky packet dropout issues caused by large-sized data packets and hard-latency constraints, usually requiring significant reliance on transmission power resources for support. This paper proposes an accelerated constrained reinforcement learning (ACRL)-based energy-efficient power scheduling (EEPS) algorithm for XR transmission to ensure a satisfactory packet dropout rate for each user with minimal power consumption. Based on a actorcritic framework, the proposed method employs a policy network to make energy-efficient power scheduling decisions online, with the critic module evaluating it through interactions with the environment, and the actor module optimizing it using the constrained stochastic successive convex approximation (CSSCA) method, which is particularly suitable for addressing non-convex stochastic constraints related to packet dropout rates. Moreover, we integrate a transfer learning method, policy reuse, into the actor module and apply a signal-reshaping mechanism to transform sparse delayed packet dropout signals into dense signals. Both techniques considerably accelerate the convergence speed of traditional constrained reinforcement learning (CRL) algorithms in XR scenarios. Simulation results demonstrate that the proposed ACRL-EEPS algorithm outperforms advanced baselines in both power conservation and meeting packet dropout constraints.