Scalable Omnidirectional Video Coding for Real-Time Virtual Reality Applications

Virtual reality (VR) can provide users an immersive and realistic visual experience, which leads to the widely use of VR in many fields. However, transmission of the ultrahigh resolution omnidirectional video requires huge bandwidth, which brings great challenges for real-time VR application. In this paper, we propose a scalable omnidirectional video coding method to improve the coding efficiency with the help of the viewer's point of view (POV) and provide three-layer scalability as well. Based on the equirectangular projection (ERP), a down-sampling procedure of ERP video with corresponding super-resolution method is proposed to save bandwidth and provide spatial resolution scalability. With the super-resolution version of the reconstructed down-sampled video as the inter-view reference, the viewer's POV within sphere is mapped and encoded in high quality, while the non POV areas are compressed in low quality to further improve the coding efficiency and provide quality scalability. The correlation of the ERP and cube map projection is utilized in the POV mapping procedure. The proposed scalable coding method is achieved based on the multiview extension of high efficiency video coding, where only a few modifications are operated in the encoder side. Experiments results demonstrate that the proposed method can save approximately 75% average bit rate with no significant decrease in quality of the viewer's POV region compared with HEVC standard.