Active Sampling for Subjective Video Quality Assessment

This paper presents an active sampling framework to achieve economic and robust subjective Video Quality Assessment (VQA). To overcome the main drawback of paired comparison that the number of pairs grows exponentially with the number of videos under test, the proposed methodology does not require the participants to perform the complete comparison of all the paired videos. Instead, we first ask some participants to perform random sampling of all possible paired comparisons. With a sufficiency of coverage satisfied motivated by Erdos-Renyi random graph, HodgeRank may give reliable results that can be used to pick out confusing pairs. Subsequently, participants will only need to commit to these confusing ones thus could save much time and labor. In other words, our interest is minimizing the number of pairs needed to learn the ranking. We demonstrate the effectiveness of the proposed framework on LIVE Database and experimental results show that it is a promising and applicable method for efficient subjective VQA.