A Long-Term-Planning Learning Strategy to Coordinate Viewport Prediction and Video Transmission in 360° Video Streaming

Fueled by Metaverse, 360° video streaming has seen tremendous growth in the past years. However, our measurement reveals that current 360° streaming systems suffer from a dilemma that severely limits QoE. On the one hand, viewport prediction requires the shortest possible prediction distance for high predicting accuracy; On the other hand, video transmission requires more buffered data to compensate for bandwidth fluctuations otherwise substantial playback rebuffering would be incurred. There is so far no existing method that can break this dilemma so the QoE optimization for 360° video streaming was naturally bottlenecked. This work is the first attempt to tackle this challenge by developing QUTA – a novel learning-based streaming system. Specifically, according to our measurement, three kinds of internal streaming parameters have significant impacts on the prediction distance, namely, download pause, data rate threshold, and playback rate. On top of this, we design a new long-term-planning (LTP) continuous control deep reinforcement learning method that tunes the parameters dynamically based on the network condition and the streaming context. Extensive evaluations based on real system prototypes show that QUTA not only improves the prediction accuracy and QoE performance by up to 68.4% but also exhibits strong temporal and spatial robustness.