EyeQ: A crowdsourcing study of subjective rate-distortion trade-offs for progressive image delivery on mobile and PC Platforms

Benchmark databases with subjective human opinions on image quality and rate-distortion trade-offs are vital to the development of Image Quality Assessment (IQA) models. Existing databases are typically built on and for PC platform. We present EyeQ, the first IQA benchmark
¹
that provides crowd-sourced subjective opinions on rate-distortion trade-offs of compressed images on both mobile and the PC platforms. Due to the increasing importance of progressive delivery of web applications, we focused our study on the use case of progressive image delivery and the associated distortions. Our results show that human perception of the perceived rate-distortion trade-off for the same image changes between PC and mobile platforms. We took 7 state-of-the-art full reference (FR) image quality assessment algorithms and evaluated their performance on predicting the optimal rate-distortion trade-off threshold for a given reference image. Our experimental results show that (a) no single FR model captures the optimal rate-distortion trade-offs across all human participants, and (b) fusion ML models that combine all 7 FR measures perform better than any individual FR model on both PC and mobile platforms.