rPPG Estimation: Vision Transformer With 3-D Temporal Central Difference

Remote photoplethysmography (rPPG) has gained increasing importance, especially during and after the COVID-19 pandemic, for its ability to estimate heart rate (HR) by analyzing subtle variations in skin color without physical contact. This noninvasive and practical method relies on capturing changes in pixel intensity through RGB video or near-infrared imaging. In this study, we propose a novel hybrid model that leverages the feed-forward integration of 3-D convolutional neural networks (3-D-CNNs) and video vision transformers (ViViTs) to enhance rPPG estimation. The 3-D-CNNs first capture local spatiotemporal features, using temporal central difference convolution (3DCDC-T) and convolutional block attention module (CBAM). These local features are then passed to the ViViT, where multihead self-attention (MHSA) captures global contextual relationships and long-range dependencies across frames, enabling a more effective representation of complex temporal dynamics. This sequential learning allows the model to progressively refine features from local to global, ensuring more consistent and coherent feature extraction. Our feed-forward approach also improves computational efficiency by reducing the dimensionality of the input data before global attention processing, making it particularly effective in data-limited environments. Through comprehensive experiments, we show that our hybrid approach outperforms state-of-the-art methods across multiple public datasets, achieving a 22.55% improvement in mean absolute error (MAE) and a 55.80% improvement in root mean-squared error (RMSE) on the UBFC-rPPG dataset, demonstrating superior feature progression and generalization in rPPG and HR estimation tasks.