Pro-UIGAN: Progressive Face Hallucination From Occluded Thumbnails

In this paper, we study the task of hallucinating an authentic high-resolution (HR) face from an occluded thumbnail. We propose a multi-stage Progressive Upsampling and Inpainting Generative Adversarial Network, dubbed Pro-UIGAN, which exploits facial geometry priors to replenish and upsample (
$8\times $
) the occluded and tiny faces (
$16\times 16$
pixels). Pro-UIGAN iteratively
(1)
estimates facial geometry priors for low-resolution (LR) faces and
(2)
acquires non-occluded HR face images under the guidance of the estimated priors. Our multi-stage hallucination network upsamples and inpaints occluded LR faces via a coarse-to-fine fashion, significantly reducing undesirable artifacts and blurriness. Specifically, we design a novel cross-modal attention module for facial priors estimation, in which an input face and its landmark features are formulated as queries and keys, respectively. Such a design encourages joint feature learning across the input facial and landmark features, and deep feature correspondences will be discovered by attention. Thus, facial appearance features and facial geometry priors are learned in a mutually beneficial manner. Extensive experiments show that our Pro-UIGAN attains visually pleasing completed HR faces, thus facilitating downstream tasks,
i.e.,
face alignment, face parsing, face recognition as well as expression classification.