Reflection Removal via Recurrent Learning Guided by Physics Prior and Focal Perceptual Loss

Removing reflection from a single image is an ill-posed problem, while exploiting physics priors can ease this inverse problem. In this paper, we integrate a physics prior of reflection-free images derived from flash illumination into deep learning. The algorithm first estimates an approximation of the transmission scene (i.e., flash-only image) from a pair of images captured with and without flash illumination. We design two collaborative neural networks to make recurrent recovery of the transmission and reflection scenes at increasing resolutions under the guidance of the physics prior. The neural networks learn the cues for scene separation and reconstruction by embedding multi-scale feature extraction components into a nested topology. We also propose a focal perceptual loss for penalizing the artifacts in output images, where the perceptual distances computed in different feature spaces are weighted adaptively to emphasize the hard-to-restore visual attributes. The comparative experiments demonstrate that the proposed algorithm performs better than the state-of-the-art methods in real-world scenarios, and it outperforms the currently best-performing algorithm by 1.5 dB in PSNR. To understand the mechanism behind the performance enhancement brought by the physics prior, we use the attribution-based model interpretation approach to quantify the pixel-to-pixel influence of the flash-only image on the results of reflection removal. The results of model interpretation reveal that the physics prior plays a significant role in dealing with non-uniform and strong reflections.