QoE Aware Transcoding for Live Streaming in SDN-Based Cloud-Aided HetNets: An Actor-Critic Approach

With the advances in hand-held devices (smart-phones and tablets, etc.) and high speed wireless networks, users have an explosive growth demand for live streaming service. Due to the diversity of user equipments (UEs), the live streaming has to be transcoded as different versions. However, transcoding is a computationally expensive and time consuming process. Since the shortage of computational resources and unstable of wireless networks, providing strict delay requirement and high quality live videos for wireless UEs is a big challenge. In this paper, we investigate user scheduling, transcoding decision, computational and wireless spectrum resources allocation problem in software-defined networking (SDN) based cloud-aided of heterogeneous networks (HetNets). Our research focuses on improving UEs' quality of experience (QoE) while guaranteeing time-delay requirement for live streaming services. Different from existing literature, to approach the real wireless environment, the available computational and wireless spectrum resources are modeled as random processes in our research. Considering dynamic characteristics of wireless networks and the available resources, the above problem is modeled as a Markov decision problem (MDP). Since the action space of the MDP is multi-dimensional continuous variables mixed with discrete variables, traditional learning algorithms are powerless. Therefore, an online actor critic algorithm is proposed to resolve the problem. Simulation results show the proposed algorithm has superior performances compared with the policy gradient algorithm and deep Q-learning network (DQN).