Spatio-Temporal Feature Integration for Quality Assessment of Stitched Omnidirectional Images

Stitched omnidirectional images are created by capturing overlapping images and seamlessly merging them together. When observing stitched omnidirectional images, users wear head-mounted display that encompass their entire visual field and are positioned near the eyes. The presence of significant distortion in the presented scene can lead to pronounced physical and mental discomfort. Therefore, it becomes crucial to explore methods for effectively evaluating the quality of stitched omnidirectional images. Given that the process of observing stitched omnidirectional images deviates from that of traditional images, resembling more closely the experience of watching videos, we posit the existence of correlation between adjacent viewport images during the viewing process. Accordingly, we have proposed a Spatio-Temporal Feature Integration Network (STFINet) that incorporates this correlation from both global and local perspectives, simulating it in the form of pseudo-videos. The STFINet utilizes 3D convolutional neural networks to extract feature vectors and integrates optical flow to capture pixel mobility among adjacent viewport images. Furthermore, we introduce a temporal hysteresis model to handle abrupt transitions. Our experimental results demonstrate the superior performance and effectiveness of STFINet in evaluating the quality of Stitched Omnidirectional images.