EmuStream—An End-to-End Platform for Streaming Video Performance Measurement

Cord-cutting has spread like wildfire as streaming video become commonplace in the Internet. This motivated intensive research in adaptive video streaming to improve its quality-of-experience (QoE) in the presence of network quality variations. Much of the existing research either employed dummy video contents or open source videos for QoE evaluation which may not capture the full spectrum of characteristics of real-world contents. This work fills this gap by developing a novel EmuStream platform based on real-world contents to enable realistic experiments and evaluation of any adaptive streaming algorithm in any streaming platform. First, EmuStream offers the largest (700+ titles) publicly available video bitrate trace dataset derived from real-world video contents. Second, we developed a mathematical model based on the dataset which can generate bitrate trace data for arbitrary target bitrates without actual video encoding. Third, we developed a novel Virtual Video Generator (VVG) which can generate h.264/h.265-compliant virtual videos that share the same frame/segment sizes as the source videos to enable experiments mimicking the streaming of commercial video contents. Last but not least, we developed a novel Streaming Performance Meter (SPM) that can measure detailed frame-by-frame playback performance from a video recording of the streaming playback of a virtual video. By decoding the virtual video's specially-coded frames, SPM can determine the playback timing and bitrate selected for each frame for use in computing any desired QoE metric. This paper presents the EmuStream platform and validates the trace data estimation model and VVG/SPM tools via controlled experiments.