Adaptive Bitrate Algorithms via Deep Reinforcement Learning With Digital Twins Assisted Trajectory

Adaptive bitrate (ABR) algorithms based on deep reinforcement learning (RL) can continuously improve their adaptability to network conditions. Existing methods usually adopt a reward function to train the ABR policies. However, in some network conditions, the quality of experience (QoE) performances obtained from the deterministic reward functions are not consistent with the users' perception. In this paper, we propose a novel learning-based deep reward model from video trajectory preference data to obtain a dynamic reward function that better reflects users' preference of the video stream. To augment the training data with more precise aggregation in a quick way, we leverage the digital twin (DT) technology to map the users' trajectory into virtual space. The digital twin can mimic users' video trajectory preferences which is used to train a reward model that can precisely predict QoE. Integrated with this reward model, the RL algorithm can generate an ABR policy that optimizes the user-perceived QoE. Experiment results show that the accuracy of the proposed reward model outperforms state-of-the-art reward functions by 13.6% in preference prediction. Compared with other RL-based ABR policies, our RL-based trajectory-preferred ABR algorithm can increase the average user QoE by 16.4%.