A Compensation Network With Error Mapping for Robust Remote Photoplethysmography in Noise-Heavy Conditions

Remote photoplethysmography (rPPG) measurement has received much attention due to its attractive applications in contactless physiological monitoring. It calculates the blood volume pulse from color signals with a normal camera. Artifacts caused by motion and illumination changes must be accounted for, especially in real-world usage. In this article, we propose a novel framework that includes an error compensation neural network after conventional signal-based rPPG extraction to improve the measuring results. Our main idea is to leverage facial image factors and frequency spectrum features to characterize the relevance between the current measurement conditions and measurement error; given inaccurate measurements, the compensation network computes the corresponding correction term to improve performance. The implementation of this compensation network includes a feature extractor based mainly on 1-D convolution kernels and a recurrent structure to account for temporal interference. Results of experiments conducted on two open datasets and two self-constructed datasets recorded for compact car drivers and passengers with driving disturbance demonstrate significant improvements over state-of-the-art methods, especially in challenging cases. The percentage of the mean absolute error with respect to that based on conventional signal-based methods improves by up to 72.3% for drivers and 66.7% for passengers.