TranPulse: Remote Photoplethysmography Estimation With Time-Varying Supervision to Disentangle Multiphysiologically Interference

Physical signs of the body that are too subtle to be observed by human eyes can reflect significant health indicators. Although many vision-based approaches have been devoted to recovery, they topically focus on recognizing explicit features such as colors, textures, and patches, and pay less attention to the entanglement and disentanglement among implicit biological characteristics. Meanwhile, existing deep networks for remote physiological detection are generally weak or deliberately neglectful in eliminating long-term time-varying interference and noise. To address these issues, we propose TranPulse, a novel remote estimation paradigm dedicated to video transformer-based polyphysiological disentanglement for robust heart rate (HR) prediction. Specifically, we improve existing single-stage transformer-based cardiac estimation backbones into a targeted two-stage architecture, which consists of a pair of asymmetric encoder and decoder. The encoder, which is the important module for heart waveform prediction, guides global attention to spatio-temporal enhancement and perception of periodic signals according to facial frame differences. The decoder, which is the core strategy pipeline for unwrapping time-varying disturbances, uses waveform gradient variations as the constraint for high-dimensional representations to separate spikes from multiple uncorrelated obstructions. Simultaneously, we integrate the synchronous computation of the encoder’s fusion of appearance and frame difference to provide more detailed guidance for in-the-wild spatio-temporal modeling, and design a more reliable regression loss function to coordinate long-term temporal and frame-wise spatial supervision. We train, validate, and practice our proposed model on multiple publicly available datasets, where it achieves competitive performance in pulse estimation by extensive experimental results.