Improving Light Field Reconstruction from Limited Focal Stack Using Diffusion Models

Light field (LF) imaging has various a pplications in image processing and computer vision. There has been growing interest in reconstructing LF from focal stack (FS) images, as directly capturing LFs with a camera array or micro-lens array has a trade-off between spatial and angular resolution in LFs. However, accurate LF reconstruction using limited noisy FS is challenging since it is a highly ill-conditioned/posed inverse problem. This paper proposes the first diffusion model based LF reconstruction approach using FS, specifically, Denoising Diffusion LF Reconstruction Model (DDLFRM). We propose 1) a new tractable system matrix decomposition method for LF reconstruction from FS to use Denoising Diffusion Restoration Model updates, and 2) a LF characteristics based U-Net architecture for LF generation. Numerical results show that the proposed DDLFRM framework achieves significantly improved LF reconstruction accuracies compared with state-of-the-art/existing LF reconstruction methods.