Three-Branch Network for Multi-Scale Spatiotemporal Feature Fusion in Remote Physiological Measurement

Remote photoplethysmography (rPPG) is a non-contact technique for measuring physiological signals from facial videos, offering significant potential in various applications. Recent advances leveraging multi-scale feature fusion have shown superior performance in extracting physiological signals. However, many existing methods focus independently on either the temporal or spatial dimensions, overlooking the inherent correlations between these dimensions in video data. To address this gap, we propose a novel three-branch architecture, termed TBNet, for rPPG signal measurement. TBNet employs fast stream, global stream, and local stream to independently capture multi-scale facial video features across temporal and spatial dimensions, followed by their fusion for robust physiological signal extraction. To further enhance feature representation and mitigate the impact of complex noise, we introduce a feature enhancement module. Additionally, we design a hybrid loss function that simultaneously optimizes performance in both the temporal and frequency domains based on the three-branch structure. Experimental results on three public datasets demonstrate that TBNet achieves state-of-the-art performance in the heart rate (HR) estimation task, achieving MAE/RMSE values of 0.16/0.40 on UBFC-rPPG, 0.47/0.69 on PURE, and 0.51/1.39 on COHFACE.