SparsePPG: Towards Driver Monitoring Using Camera-Based Vital Signs Estimation in Near-Infrared

Camera-based measurement of the heartbeat signal from minute changes in the appearance of a person's skin is known as remote photoplethysmography (rPPG). Methods for rPPG have improved considerably in recent years, making possible its integration into applications such as telemedicine. Driver monitoring using in-car cameras is another potential application of this emerging technology. Unfortunately, there are several challenges unique to the driver monitoring context that must be overcome. First, there are drastic illumination changes on the driver's face, both during the day (as sun filters in and out of overhead trees, etc.) and at night (from streetlamps and oncoming headlights), which current rPPG algorithms cannot account for. We argue that these variations are significantly reduced by narrow-bandwidth near-infrared (NIR) active illumination at 940 nm, with matching bandpass filter on the camera. Second, the amount of motion during driving is significant. We perform a preliminary analysis of the motion magnitude and argue that any in-car solution must provide better robustness to motion artifacts. Third, low signal-to-noise ratio (SNR) and false peaks due to motion have the potential to confound the rPPG signal. To address these challenges, we develop a novel rPPG signal tracking and denoising algorithm (sparsePPG) based on Robust Principal Components Analysis and sparse frequency spectrum estimation. We release a new dataset of face videos collected simultaneously in RGB and NIR.We demonstrate that in each of these frequency ranges, our new method performs as well as or better than current state-of-the-art rPPG algorithms. Overall, our preliminary study indicates that while driver vital signs monitoring using cameras is promising, much work needs to be done in terms of improving robustness to motion artifacts before it becomes practical.