Vaser: Optimizing 360-Degree Live Video Ingest via Viewport-Aware Neural Enhancement

As a revolutionary technique, 360-degree live video streaming provides users with an immersive and realistic experience for a live event. However, due to the limited upload bandwidth of broadcasters, the ingest of 360-degree live video streams has become a bottleneck that seriously lowers the overall utility of viewers. In this paper, we first conduct a series of measurement studies to unveil the characteristics of 360-degree video viewers and the effectiveness of neural enhancement under various settings. Based on the obtained insights, we further propose a novel neural-enhanced 360-degree live video ingest framework called Vaser. A key innovation of Vaser is to take viewport information into account when conducting neural enhancement and tile uploading. Vaser can significantly reduce the upload bandwidth demand for 360-degree live video ingest without sacrificing user experiences. We also propose an enhanced complexity-based patch selection algorithm to improve the efficiency of SR model training. To better utilize the upload bandwidth, we exploit deep reinforcement learning (DRL) to determine the bitrate of upload streams and the frequency of SR model updates. Last, we implement the whole framework and validate its effectiveness using real network traces. The experimental results show that our proposed framework can reduce the upload bandwidth consumption by 40%-55.6% on average with an upscale ratio of 2 and improve the overall utility of viewers by 1.15x-3.61x compared to other baseline approaches.