Stacked Hourglass Network Joint with Salient Region Attention Refinement for Face Alignment

Localizing facial landmarks is a fundamental step in facial image analysis. However, the problem continues to be challenging in condition of large variations caused by pose disparity, illumination, expression and occlusion. In this paper, we propose a coarse-to-fine framework which joints stacked hourglass network and salient region attention refinement for robust face alignment. To achieve this, we firstly develop a multi-scale region learning module (MSL) to analyze the structure and texture information at different facial region and extract strong discriminative deep feature. Then we employ a novel convolutional neural network named stacked hourglass network (SHN) for heatmap regression and initial facial landmarks prediction. Moreover, we present a salient region attention module (SRA) to extract precise feature based on the heatmap regression, and the filtered feature is used for landmarks refinement. The extensive experimental results on two public datasets, including 300W and COFW, confirm the validity of our model.

I. INTRODUCTION

Face alignment, also known as facial landmarks prediction, gets at the task of calculating the coordinates of several predefined points on a facial image, including eyes, nose, mouth, chin, etc. As shown in [1], [2], [3], accurate face alignment improves the performance of face verification system. Moreover, face alignment becomes the most pivotal component in many other facial analysis assignments, e.g. facial expression classification [4], [5] and 3D face modeling [6], [7]. In the past decades, many researches and algorithms have been developed for face alignment, it remains a challenging task, especially when the faces suffer from the huge variations caused by arbitrary pose and partial occlusions.