Depth-Gyro Sensor-Based Extended Face Orientation Estimation Using Deep Learning

Conventional face direction estimation techniques detect the characteristic parts of the face, such as the nose, eyes, and mouth, and estimate the face orientation based on the movements of these features. However, these methods cannot accurately estimate the face direction when the characteristic parts of the face are hidden; for example, when the face is turned sideways or a mask is worn. Face detection using point cloud data has been investigated as a solution to these issues. Previous studies applied five classes of face direction estimation for the head using 3-D point cloud data. However, considering the practical use of driver assistance systems that verify the driver’s status, these five classes are not sufficient for accurately detecting the face direction, and a more precise horizontal wide-range angle detection approach is necessary. In this study, we acquired 3-D point cloud data in
${k}$
(where
${k} >$
5) classes while accurately measuring the horizontal angle of the face during the acquisition of the training data using gyroscopic sensors. The training data captured by this depth-gyro sensor integration generates accurate depth data for each direction. As a result, a low number of point cloud data samples for each face direction were sufficient for generating the directional classification model. Therefore, this depth-gyro sensor integrated data capturing significantly reduces the amount of required training data. Furthermore, we applied a weight reduction process for the point cloud data to reduce the training time and performed deep learning to estimate the face direction. The proposed method achieved high performance in face direction detection using deep learning, even with a comparatively small dataset.