On the Impact of COVID-19 on Subjective Digital Media Quality Assessment

The COVID-19 pandemic has induced dramatic effects in all areas of society worldwide. It has led to drastic restrictions on academic exchanges and caused research projects to be put on hold. The need to maintain social distancing has severely affected the research communities that rely on in-person studies. In particular, Quality of Experience (QoE) research relies heavily on user studies and in-person subjective tests as means of obtaining ground truths for system designs. In this position paper, we focus on the impact of COVID-19 on conducting subjective tests for digital media quality assessment. An overview of the related discussions that take place in associated research communities is provided. A number of challenges are posed in terms of hygiene, ethics, standardization, and research quality issues. Opportunities for consolidating QoE research under COVID-19 and beyond are put forward, including alternative experimental designs and adaptation of research methodologies. Numerous enablers are suggested and discussed, revealing various avenues that can be followed to assist in conducting subjective tests under pandemic conditions. We believe that this position paper will be helpful for researchers and practitioners relying on in-person studies to keep abreast about the potential measures that support coming out of the COVID-19 pandemic strong and being prepared for future endeavors.