Light Field Editing Propagation using 4D Convolutional Neural Networks

2D image editing has been a well-studied problem. However, 2D image processing techniques cannot be directly applied to the emerging light field image (LFI) due to the particular structural characteristics of LFI. Without a dedicatedly designed editing scheme for LFI, users need to manually edit each sub-view of the LFI. This process is extremely time consuming, and more importantly, users have no ways to guarantee parallax consistency between sub-views. This poster proposes two different LFI editing schemes including the direct editing scheme and the deep-learning-based scheme. These schemes enable automatic propagation of the user’s edits, particularly “augmentation” editing, from central view to all the other sub-views of LFI. In particular, the learning-based scheme employs interleaved spatial-angular convolutions (4D CNN) to enable effective learning of both spatial and angular features, which are subsequently used to help the augmentation editing. We constructed a preliminary LFI dataset and compared the proposed two schemes. The experimental results show that the learning-based scheme produces higher PSNR (0.51dB) and more pleasant subjective editing results than the direct editing.