3DGStreaming: Spatial Heterogeneity Aware 3D Gaussian Splatting Compression and Streaming

3D Gaussian Splatting (3DGS) has emerged as a promising technique for high-quality 3D scene representation. However, streaming 3DGS scenes poses significant challenges due to large data volumes and complex spatial structures, resulting in non-smooth scene loading, inferior visual quality, and ineffective streaming adaptation, ultimately impacting user experience adversely. To tackle these challenges and enhance user Quality of Experience (QoE), this paper introduces a novel adaptive streaming framework called 3DGStreaming. Our framework comprises three key components: (1) Smart Spatial Partitioning for efficient scene division, enabling selective streaming and seamless scene merging; (2) Two-step Progressive Scene Generation, involving content-aware downsampling and attribute compression to create multi-bitrate 3DGS scenes; and (3) FoV-based Bitrate Adaptation using a decision transformer for viewport-based bitrate selection. Extensive experiments demonstrate the superiority of 3DGStreaming over existing state-of-the-art solutions. 3DGStreaming achieves a greater rendering quality with a 5.7% -25.5% increase in PSNR, a 27.8% -69.2% reduction in latency, a 54.7% reduction in training time, and a 17.2% -68.3% improvement in overall QoE.