GRadient sharing A3C for Adaptive Bitrate (GRAAB): A DRL Based Approach for Adaptive Video Streaming

Recent research has shown that the deep reinforcement learning (DRL) based asynchronous advantage actor-critic (A3C), an on-policy method, is an effective tool for generating adaptive bit rates (ABR) for video streaming and provides a high quality of experience (QoE) for end users under varying network conditions. However, vanilla A3C suffers from the following issues: (i) it has biased estimates due to the use of high entropy weights, (ii) it has a high number of computations due to the use of near-homogeneous policies while training with a single set of global parameters with multiple agents, and (iii) it lacks exploration due to the use of the on-policy method. To address the aforementioned issues, this paper presents gradient sharing A3C for adaptive bitrate (GRAAB) to generate adaptive bitrates. By sharing gradients and parameters among multiple workers, GRAAB encourages exploration and produces optimal policies without compromising long-term convergence. Using 5G traces from the Lumos 5G dataset, we highlight the advantages of GRAAB and demonstrate that it achieves a significantly higher average QoE than a number of other state-of-the-art ABR algorithms.