Locality constraint neighbour embedding via Reference Patch

Recently, face hallucination (FH) methods using position priors have gained popularity; however position priors might not always be the best due to the intrinsic rigidness of faces collected from uncontrollable environment. Therefore, we improve the search criteria for K-nearest neighbors (K-NN) to address the variations in human facial features. Meanwhile, the limitations of the manifold assumption are taken into consideration to refine the neighborhood of the low-resolution (LR) patch by using the information from the high-resolution (HR) patches. For each input patch, we search the local neighborhood of its corresponding position patch in each training image to find the best-matched neighbor “Reference Patches”. Reference patches and their HR counterparts are taken to construct LR and HR patch dictionaries. The proposed method is composed of two steps. For an input LR patch, first we construct its initial HR patch using conventional FH methods. Secondly, we search the initial HR patch's nearest neighbors in HR manifold to extract the discriminant locality constraints. Then the corresponding LR reference patches are taken as refined K-NN of the input patch. These refined reference patches better optimize the reconstruction weights, thus the performance is improved. Extensive experiments show that our method outperforms recent position patch schemes in reconstruction error and visual quality.