Spatial Quality Oriented Rate Control for Volumetric Video Streaming via Deep Reinforcement Learning

Volumetric videos offer an incredibly immersive viewing experience but encounters challenges in maintaining quality of experience (QoE) due to its ultra-high bandwidth requirements. One significant challenge stems from user’s spatial interactions, potentially leading to discrepancies between transmission bitrates and the actual quality of rendered viewports. In this study, we conduct comprehensive measurement experiments to investigate the impact of six degrees of freedom information on received video quality. Our results indicate that the correlation between spatial quality and transmission bitrates is influenced by the user’s viewing distance, exhibiting variability among users. To address this, we propose a spatial quality oriented rate control system, namely sparkle, that aims to satisfy spatial quality requirements while maximizing long-term QoE for volumetric video streaming services. Leveraging richer user interaction information, we devise a tailored learning-based algorithm to enhance long-term QoE. To address the complexity brought by richer state input and precise allocation, we integrate pre-constraints derived from three-dimensional displays to intervene action selection, efficiently reducing the action space and speeding up convergence. Extensive experimental results illustrate that sparkle significantly enhances the averaged QoE by up to 29% under practical network and user tracking scenarios.