Head Pose-Aware Regression for Pupil Localization From a-Pillar Cameras

In vehicular applications, remote eye pupil tracking is essential, particularly for advanced augmented reality (AR) 3D head-up displays (HUDs), and driver monitoring systems (DMS). However, achieving accurate pupil center localization under varied head poses presents significant challenges, especially when cameras are positioned on a vehicle’s A-pillar. This placement introduces substantial head pose variations, complicating traditional tracking methods. In response, this study presents a remote pupil localization method designed to address the unique challenges posed by a camera situated on a vehicle’s A-pillar, a spot causing significant head pose variations. The proposed technique relies on a head pose-aware pupil localization strategy utilizing A-pillar cameras. Our pupil localization algorithm adopts a Transformer regression approach, into which we integrate head pose estimation data, enhancing its capability across diverse head poses. To further enhance our approach, we used an optimized nine-point eye-nose landmark set, to minimize the pupil center localization loss. To demonstrate the robustness of our method, we conducted evaluations using both the public WIDER Facial Landmarks in-the-wild (WFLW) dataset and a custom in-house dataset focused on A-pillar camera captures. Results indicate a Normalized Mean Error (NME) of 2.79% and a failure rate (FR) of 1.28% on the WFLW dataset. On our in-house dataset, the method achieved an NME of 2.96% and a FR of 0.72%. These impressive results demonstrate the robustness and efficacy of our method, suggesting its potential for implementation in commercial eye tracking systems using A-pillar mounted cameras, especially for AR 3D HUD and DMS applications.