AIFNet: All-in-Focus Image Restoration Network Using a Light Field-Based Dataset

Defocus blur often degrades the performance of image understanding, such as object recognition and image segmentation. Restoring an all-in-focus image from its defocused version is highly beneficial to visual information processing and many photographic applications, despite being a severely ill-posed problem. We propose a novel convolutional neural network architecture AIFNet for removing spatially-varying defocus blur from a single defocused image. We leverage light field synthetic aperture and refocusing techniques to generate a large set of realistic defocused and all-in-focus image pairs depicting a variety of natural scenes for network training. AIFNet consists of three modules: defocus map estimation, deblurring and domain adaptation. The effects and performance of various network components are extensively evaluated. We also compare our method with existing solutions using several publicly available datasets. Quantitative and qualitative evaluations demonstrate that AIFNet shows the state-of-the-art performance.

I. Introduction

Out-Of-Focus or defocus blur occurs when the rays emitting from a scene point not lying on the focal plane of the camera converge to a region rather than a point on the image plane. This region is called circle of confusion (COC) [1]. Using a larger aperture can allow an increased amount of light to reach the image sensor within a shorter time, the depth of field (DOF) is however reduced, thereby causing more defocus blur. Shallow DOF is sometimes an artistic effect purposely created by the photographer to make the subject stand out from the blurry background and foreground. However, defocus blur is undesirable for most computer vision and image processing tasks. Defocused images lose key information in the blurred regions, thus obstructing the algorithms for image understanding [2], [3]. Restoring all-in-focus images from defocused ones can facilitate a wide range of applications, including object recognition, face detection, image segmentation, image stitching and misfocus correction.