HREADAI: Heart Rate Estimation From Face Mask Videos by Consolidating Eulerian and Lagrangian Approaches

Heart rate (HR) is a crucial human vital parameter that can provide essential information about an individual’s physical and physiological well-being. The COVID-19 pandemic has proliferated the requirement for noninvasive HR monitoring techniques such as remote PPG (rPPG). Ironically, it also brings widespread use of face masks that posed a challenge to rPPG because most facial skin is covered. The existing works utilize nonoccluded face regions but neglect the occluded face regions for extracting relevant rPPG information, thereby providing limited performance. Hence, our proposed rPPG-based HR estimation method, Heart Rate Estimation from face mAsk viDeos by consolidating EuleriAn and LagrangIan (HREADAI), investigates the possibility of extracting relevant rPPG information from the mask region. To this end, the Lagrangian approach is investigated to leverage the intuition that the movements of the mask placed on the face are similar to that of the facial movements. The HREADAI further improves the performance by quality-based consolidation of relevant rPPG information extracted from the forehead and mask region using the Eulerian and Lagrangian approaches, respectively. Since no dataset is publicly available for this rPPG estimation of face mask videos, the experiments are performed by collecting a novel dataset comprising face mask videos and their pulse rate information. It demonstrates that our method, HREADAI, successfully extracts relevant rPPG information from mask regions and outperforms the existing rPPG-based HR estimation methods for face mask subjects. Hence, this article provides a significant step toward rPPG-based HR estimation for face mask subjects. The code and dataset will be made publicly available upon paper acceptance.