Non-Contact Heart Rate Detection Based on Spatio-Temporal Hybrid Deep Learning

To enable low-latency non-contact heart rate detection in natural scenarios, this paper proposes a dual-buffer spatio-temporal hybrid deep learning network. Firstly, an ordinary camera is used to capture facial videos, which are then processed using a two-queue buffering and multi-channel input approach before being fed into a spatial feature extraction network. The network works through feature extraction, spatial self-attention, feature integration, and prediction to obtain the rPPG (remote photoplethysmography) signal sequence. This sequence is subsequently passed through an LSTM network for temporal feature extraction of heart rate, ultimately predicting the corresponding heart rate value. This method can predict both rPPG signals and heart rate values in a multitasking manner. Experimental results demonstrate that on the UBFC-rPPG dataset, the Pearson correlation coefficient for rPPG signals is 0.90, and the root mean square error (RMSE) for heart rate values is $\mathbf{3. 8 9}$. Compared to reference models, this method improves the correlation of rPPG signals and reduces the RMSE of heart rate values. The proposed model exhibits lower latency and better robustness.