Towards subjective quality of experience assessment for omnidirectional video streaming

Currently, we witness dramatically increasing interest in immersive media technologies like Virtual Reality (VR), particularly in omnidirectional video (OV) streaming. Omnidirectional (also called 360-degree) videos are panoramic spherical videos in which the user can look around during playback and which therefore can be understood as hybrids between traditional movie streaming and interactive VR worlds. Unfortunately, streaming this kind of content is extremely bandwidth intensive (compared to traditional 2D video) and therefore, Quality of Experience (QoE) tends to deteriorate significantly in absence of continuous optimal bandwidth conditions. In this paper, we present a first approach towards subjective QoE assessment for omnidirectional video (OV) streaming. We present the results of a lab study on the QoE impact of stalling in the context of OV streaming using head-mounted displays (HMDs). Our findings show that subjective testing for immersive media like OV is not trivial, with even simple cases like stalling leading to unexpected results. After a discussion of characteristic pitfalls and lessons learned, we provide a a set of recommendations for upcoming OV assessment studies.

I. Introduction

Universal access to and provisioning of (multi-)media content have become reality, nowadays it is very easy-in realtime-to generate, distribute, share, and consume any media content, anywhere, anytime, and with any device. These kind of real-time entertainment services-specifically, streaming audio and video-are typically deployed over the open, unmanaged Internet and account now for more than 70% of the evening traffic in North American fixed access networks. It is assumed that this number will reach 80% by the end of 2020 [1]. Although Internet capacity is constantly increasing for both fixed and mobile networks, the adoption of streaming audio and video services will continue as well as new applications and services will emerge including-but not limited to-ultra high-definition (UHD) 4K/8K, high dynamic range (HDR), and virtual reality (VR), specifically omnidirectional video (OV) also known as 360-degree video. Consequently, the data volume increases which calls for adaptive streaming techniques to cope with bandwidth fluctuations or-in general-with dynamically changing context conditions.