Spherical Video Coding with Geometry and Region Adaptive Transform Domain Temporal Prediction

Many virtual and augmented reality applications depend critically on efficient compression of spherical videos. Current approaches apply a projection geometry to map a spherical video onto the plane(s), wherein a standard codec can be used for compression. Video coders employ simple pixel copying from reference frames for inter-prediction, which ignores underlying spatial correlations, and is hence suboptimal. A novel paradigm of transform domain temporal prediction (TDTP) was developed previously in our lab to effectively overcome this suboptimality of standard video coding. This paper is motivated by the observation that projected spherical videos exhibit significantly more statistical variation due to i) the choice of projection geometry and ii) position of the block on the sphere, which reflect variations in sampling density and various statistical features. To account for such variations, we propose geometry and region adaptive TDTP that is tailored to spherical videos. For a given geometry, the sphere is divided into regions, according to expected signal statistics, and prediction filters are designed for each region. Experimental results show significant performance gains as evidence for the efficacy of TDTP in spherical video coding.