Semi-Supervised Learning: Structure, Reflectance and Lighting Estimation From a Night Image Pair

While unsupervised approaches have been proposed to estimate reflectance and shading layers for images, the decomposition process is a challenging, under-determined inverse problem. Previous unsupervised approaches for intrinsic decomposition of images are strictly reliant on special image sequences, for example, image sequences for a fixed scene with changing illumination or multi-view images containing rich illumination variation. As an alternative to these unsupervised intrinsic decomposition methods, we propose a semi-supervised method. Our method adopts a simplified scene representation to greatly reduce the complexity of spatially varying lighting, which allows us to partially restore the lighting from a night image pair. Moreover, we design several novel unsupervised guiding losses, and the training data used in our method are easily collected by setting different exposure times for a standard stereo setup. We further demonstrate the effectiveness of the proposed method by quantitatively and qualitatively comparing our method with recent works on the modified MPI dataset and a collected highly dynamic dataset named NightCampus. The NightCampus dataset has been released at
https://github.com/kewangtt/srlefnip
.