Panoramic Image Quality-Enhancement by Fusing Neural Textures of the Adaptive Initial Viewport

With the development of virtual reality (VR) technology, panoramic image has been widely applied in our life. Due to its large size, the existing streaming methods prefer only transmitting contents corresponding to the audience’s current viewport in high quality. This viewport-based transmission, however, suffers from severe delay as the viewport changes. In this paper, to fill in the time gap between the switch of viewport and the arrival of high-resolution content, we introduce an end-to-end network at the receiver. The main idea is to use the neural textures in the adaptive initial viewport to improve the quality of regions around it. When the viewport changes but the high-resolution content has not arrived, the image enhanced by our strategy is acceptable to satisfy visual experience as the experiment results show. To the best of our knowledge, this is the first panoramic image quality-enhancement method considering content continuities and internal features.