Multiview super-resolution using high-frequency synthesis in case of low-framerate depth information

Increasing the image sharpness of low-resolution views is a key issue in the multiview image and video processing domain. Thereby, a low-resolution view gets refined by high-frequency content that can either be obtained from temporally or spatially adjacent highresolution reference images. We propose a refined super-resolution algorithm for multiview images that is robust to the usage of temporally highly misaligned depth maps. The temporal misalignment may be caused either by a temporal subsampling or a lower framerate of the depth camera with respect to the image cameras. Our refinement step is based on a blockwise low-frequency registration in order to efficiently adapt the high-frequency content of the highresolution reference to the low-resolution destination view. The simulation results show that our proposed algorithm leads to a peak PSNR gain of up to 1.21 dB with respect to a comparable unrefined super-resolution approach. On average, our approach outperforms the unrefined super-resolution algorithm by 0.61 dB. For all considered scenarios the improvement of visual quality is also convincingly.