Content-Aware Perspective Projection Optimization for Viewport Rendering of 360° Images

To visualize the spherical (360°) content on 2D planar displays, a selected part of the sphere must be projected into a 2D plane. The resulting 2D area is called viewport and is created according some fixed and limited field of view, usually much less than 360°. To generate the viewport, two geometric projections are usually employed: rectilinear and stereographic. For both cases, this process leads to geometric distortions in the viewport image which, depending on the content, may be visually unpleasant to users. Rectilinear projection stretches the image borders while the stereographic projection bends straight lines. In this paper, the viewport rendering problem is tackled with the formulation of a generalized perspective projection (which includes many other popular projection, besides perspective and stereographic) and a perceptual optimization of its key parameter: the projection center. The optimization process is performed according to the viewport characteristics, namely with perceptually important features that characterize the amount of line distortion and stretching of salient regions. The aim is to find an optimal, content dependent, projection center that minimizes the content distortions and thus provides better quality of experience (QoE) in the visualization of spherical images. Experimental results on several 360° images show that the proposed optimized projection achieves better results than the benchmark projections both quantitatively and qualitatively.