An Online Sampling Approach for Controlled Experimentation and QoE Modeling

Predicting Quality of Experience (QoE) of end users from available network Quality of Service (QoS) measurements is of significant importance for today's network and content providers. This can be achieved by using application-specific QoE models that map the network QoS to the output QoE. QoS-QoE models can be built by training supervised machine learning algorithms with training data consisting of the mappings of the input network QoS to the output QoE. In most ML works on QoE modeling, the training data is usually gathered in the wild inside the core of the service or the content provider networks. However, such data is not easily accessible to the general research community. Consequently, the training data if not available before hand, needs to be built up by controlled experimentation. Here, the fundamental challenge is the sheer amount of time consumed in collecting the datasets needed to model the QoE. Considering this problem, we present here a framework of controlled experimentation based on active learning, that allows collecting rich datasets covering the experimental space intelligently. We perform a rigorous analysis of our approach and demonstrate the performance improvement over conventional pool based uncertainty sampling for a particular use case of YouTube video streaming.