Lighting is Unreliable: Adversarial Video Relighting Against rPPG Heart Rate Measurement

Facial video-based remote physiological measurement (rPPG) has gained prominence for its ability to non-invasively estimate vital signs such as heart rate (HR). rPPG measures HR by detecting variations in the diffuse reflection of light caused by blood volume changes in the skin, which is influenced by light conditions. Inspired by this property, we identify a new task, that is, to embed malicious information into facial video by subtly altering the light conditions. With this task, we can mislead state-of-the-art rPPG HR methods through natural and imperceptible lighting changes, aiming for two objectives: testing the resilience of rPPG methods against light variations and safeguarding heart rate data, which is crucial for individual privacy. However, such a task is non-trivial and should be able to adapt to different input videos automatically and generate natural and imperceptible spatial-temporal lighting perturbations. To address these challenges, we propose the AdversariaL vidEo Relighting aTtack (ALERT) method, which involves three modules: video lighting estimation (VLightE), adversarial temporal lighting prediction (AdvTLight), and adversarial temporal lighting injection (AdvTLightInj). VLightE is to estimate the spatial-temporal lighting conditions of the original video. AdvTLight predicts the adversarial spatial-temporal lighting conditions that are imperceptible but can mislead the HR detectors according to the original lighting conditions automatically. The final module (i.e., AdvTLightInj) is to inject the predicted adversarial lighting conditions into the input video and render a new one. Extensive experiments on UBFC-rPPG and PURE datasets demonstrate that ALERT generates realistic, imperceptible adversarial videos, effectively misleading 11 rPPG-based HR methods and outperforming all baseline methods. Moreover, our method can be used to protect the HR privacy of users directly and outperform two SOTA Privacy-protection-oriented methods significantly.