Deep Learning Based Estimation of Facial Attributes on Challenging Mobile Phone Face Datasets

Facial attribute analysis is an important step in many biometric algorithms, including face recognition based human authentication. Detecting the state of face attributes is becoming even more popular in mobile based applications, where a biometric authentication application is used to quickly and accurately verify the claimed identity of the owners device, and also, keep the device secure if an intruder attempts to gain unauthorized access to it. While there is a large number of facial attributes that can be automatically detected to support a face recognition system, in this paper we focus on detecting three specific ones that are useful during both an enrollment and authentication process: (1) determining whether the eyes of a subject are open or closed, (2) determining whether a subject is wearing glasses or not, and, finally, (3) detecting whether a subjects facial pose is either frontal or non-frontal. These attributes are associated with face image quality control, which is a very useful component of modern FR systems under the following context: during live authentication, by limiting low quality face image data, we can enhance the face-based authentication accuracy, while at the same time improve user satisfaction via improved system efficiency (i.e. less false match attempts during authentication). Thus, to automatically and efficiently detect all of the aforementioned facial attributes, we developed both conventional and deep learning based models. These models are trained and tested on diverse and challenging face image datasets, using data captured from traditional cameras and mobile devices, when operating at multiple standoff distances, in either indoor or outdoor conditions. Our proposed attribute-specific detection models are robust, yielding up to 100% accuracy (in terms of F1 score) depending on the attribute tested, as well as the model and dataset(s) used for training and testing.