QoE Optimization for VR Streaming: a Continual RL Framework in Digital Twin-empowered MEC

Mobile edge computing (MEC) resource allocation for remote rendering in virtual reality (VR) content streaming is critical for user experience. However, resource allocation becomes challenging due to the desynchronization between the physical and digital worlds in digital twin-empowered MEC. This paper presents our continual RL framework that facilitates dynamic resource allocation for MEC-enabled VR content streaming. We first design a digital twin-empowered edge computing (DTEC) system and formulate a maximization problem that considers attention-based resolution perception to maximize the quality of experience (QoE). This problem optimizes the allocation of computing and bandwidth resources while adapting the attention-based resolution of the VR content. We then apply continual reinforcement learning (CRL) to enable adaptive attention-based resolution VR streaming in a time-varying environment. We base the CRL's reward function on the QoE and horizon-fairness QoE (hfQoE) constraints. We support CRL with prioritized experience replay - continual deep deterministic policy gradient (PER-CDDPG) to enhance the performance of continual learning in the presence of time-varying DT updates. We test PER-CDDPG using extensive experiments and evaluation. PER-CDDPG outperforms the benchmarks in terms of average latency, QoE, and successful delivery rate as well as meeting the hfQoE requirements and performance over long-term execution while ensuring system scalability with the increasing number of users.