Neural Reflectance Decomposition Under Dynamic Point Light

Decomposing a scene into its 3D geometry, surface material textures, and illumination is a challenging but important problem in computer vision and graphics. While recent neural implicit representation based works have shown tremendous advantages, existing methods are not applicable to images illuminated by a single dynamic point light. We propose an entirely self-supervised end-to-end neural implicit representation based reflectance decomposition algorithm for objects under a dynamic point light. Our method adopts a staged training framework to estimate the geometry, light source position, and surface material textures through volume rendering, self-shadow inverse rendering, and physical model based surface rendering respectively. This scheme allows accurate recovery of the surface material textures which are coupled to the dynamic light, improving the reflectance decomposition capability. For evaluation, we collect a new dataset of several synthetic and real world objects illuminated by a moving point light. Experiments show that our method achieves superior reflectance decomposition performance compared to state-of-the-art methods, and the recovered elements can be deployed in existing graphics pipelines to perform relighting, material editing, and scene composition.