Multi-image super-resolution for fisheye video sequences using subpixel motion estimation based on calibrated re-projection

Super-resolution techniques are a means for reconstructing a higher spatial resolution from low resolution content, which is especially important for automotive or surveillance systems. Furthermore, being able to capture a large area with a single camera can be realized by using ultra-wide angle lenses, as employed in so-called fisheye cameras. However, the underlying non-perspective projection function of fisheye cameras introduces significant radial distortion, which is not considered by conventional super-resolution techniques. In this paper, we therefore propose the integration of a fisheye-adapted motion estimation approach that is based on a calibrated re-projection into a multi-image super-resolution framework. The proposed method is capable of taking the fisheye characteristics into account, thus improving the reconstruction quality. Simulation results show an average gain in luminance PSNR of up to 0.3 dB for upscaling factors of 2 and 4. Visual examples substantiate the objective results.