A Complementary Dual-Branch Network for Appearance-Based Gaze Estimation From Low-Resolution Facial Image

Estimating gaze from a low-resolution (LR) facial image is a challenging task. Most current networks for gaze estimation focus on using face images of adequate resolution. Their performance degrades when the image resolution decreases due to information loss. This work aims to explore more helpful face and gaze information in a novel way to alleviate the problem of information loss in the LR gaze estimation task. Considering that all faces have a relatively fixed structure, it is feasible to reconstruct the residual information of face and gaze based on the solid constraint of the prior knowledge of face structure through learning an end-to-end mapping from pairs of low- and high-resolution (HR) images. This article proposes a complementary dual-branch network (CDBN) to achieve this task. A fundamental branch is designed to extract features of the major structural information from LR input. A residual branch is employed to reconstruct features containing the residual information as a supplement under the supervision of both the HR image and gaze direction. These two features are then fused and processed for gaze estimation. Experimental results on three widely used data sets, MPIIFaceGaze, EYEDIAP, and RT-GENE, show that the proposed CDBN achieves more accurate gaze estimation from the LR input image compared with the state-of-the-art methods.