Fine-Grained Blind Face Inpainting with 3D Face Component Disentanglement

Inpainting is a task to restore occlusion or other corruption on images. However, previous works require mask of the occluded area to restore the occluded image, which is inconvenient for application. Blind face inpainting aims to automatically restore the occluded face without position information of the corrupt region. In this paper, we propose a novel fine-grained blind face inpainting framework, combining 3D face components disentanglement with generative network. Canonical face texture and shape disentangled by unsupervised 3D face model is restored separately to get occlusion-free rendered result. Finally, the pixel-to-pixel generative module utilize the occlusion image and the coarse de-occlusion face to get refined inpainted result. We also build up a new dataset called CelebO-3D which consists of occluded face images synthesized with 3D occlusion and rendered by 3D face model. Extensive experiments show that the proposed method is effective and robust in face blind inpainting both in synthesized and real images. Extensive evaluations and comparison with previous methods also show our superior effectiveness and lightweight architecture.