A Q-Learning solution for adaptive video streaming

Adaptive streaming is a promising technique for video streaming service to cope with the quality degradation of mobile users' connections. In this paper we propose a service layer control mechanism for video flows based on a Reinforcement Learning (RL) paradigm that will gracefully degrade the video quality experienced by the end-user depending on the connection status. Using layered coded videos, the end-user should find the most appropriate quality level for its stream. The adaptive streaming problem can naturally be modeled as a Partial Observable Markov Decision Process (POMDP) because the end-user has partial information about the network state based on the received throughput, but this model cannot be applied on-line during streaming. We propose here an MDP modeling the adaptive streaming problem that can be solved on-line by the Q-Learning algorithm. Both models have identical solutions proving the validity of the proposed MDP model.