Toward Attention-Aware Interactive 360° Video Streaming on Smartphones

Browsing 360-degree videos on smartphones is becoming increasingly popular, which is challenging due to heavy user interactions, limited hardware capabilities, and constrained batteries. To improve resource efficiency and user experience, we propose a framework for attention-aware interactive 360-degree video streaming for smartphone users. We first explore the built-in capabilities of off-the-shelf smartphones to capture user attention during video browsing. Using the information of user attention to predict viewport and analyze user-perceptive quality, we then investigate the approaches for improving video encoding, transmission, and rendering in 360-degree video streaming. Further, we discuss the opportunities for enhancing user interactive experience with error tolerance, protocol design, and on-device computation. Finally, our prototype implementation is presented as a case study to validate our design principles. Our framework integrates and coordinates advanced techniques for different components of 360-degree video streaming systems to achieve better user experience, which can also serve as a foundation for leveraging modern attention-based multi-modal learning architectures.