Joint Upsampling for Refocusing Light Fields Derived With Hybrid Lenses

As a 3-D world sensing instrument, the light field camera provides an effective measure to perceive the depth information, which enables the fascinating function to refocus after exposure. However, existing light field cameras suffer from limited spatial resolution. Recently, hybrid imaging systems have been widely explored to overcome this limitation, where only the central sub-view (sub-aperture image) has a high resolution. In this article, we propose a novel refocus scheme for the hybrid imaging system, which derives the central-view refocused image at low resolution, then jointly upsamples it under the guidance of the high-resolution central sub-view. The state-of-the-art joint upsampling methods are based on the local-linear-relation (LLR) or the consistent-structure (CS) assumptions, which are barely valid for our application scenario. We propose a linear fusion block (LFB) and stack the proposed blocks into a lightweight fully convolutional network named linear fusion network (LFN) for refocused image upsampling. We have conducted extensive experiments to evaluate the proposed method. Both qualitative and quantitative measures are explored. Experimental results indicate that LFN outperforms existing methods on the task of refocused image upsampling and shows that the proposed refocus scheme is practical.