Motion-Robust Respiratory Rate Estimation From Camera Videos via Fusing Pixel Movement and Pixel Intensity Information

Recently, noncontact monitoring of respiratory rate (RR) based on consumer-level cameras has gained increasing attention. However, motion artifacts with large amplitudes usually contaminate subtle video-extracted respiratory signals, challenging the performance of video-based RR estimation. To address this problem, we propose to estimate RR by fusing the pixel movement source and the pixel intensity source on both feature and decision levels. Specifically, rigid motion artifacts are first separately denoised from each source by spectral subtraction (SS), while canonical correlation analysis (CCA) is then taken to further extract the underlying common respiratory signals existing within both sources. The two respiratory signals reconstructed from retaining the canonical variates (CVs) with a correlation coefficient larger than a threshold are treated as the fused results of the two sources at a feature level. Finally, the target RR value is determined by fusing all obtained RR results from both sources on a decision level, with weights calculated by signal-to-noise ratio (SNR) of each respiratory signal. The performance of the proposed two-level fusion method is evaluated on the in-house BSIPL-RR database including 30 healthy subjects, performing diverse motions while breathing during natural lighting conditions. Experimental results demonstrate the superiority of the proposed method, with the root mean square error (RMSE) up to 1.61 breaths per minute (bpm) during the rest scenario, 2.29 bpm during the walking scenario, 3.51 bpm during the running scenario and 2.96 bpm during the variety scenario. This study will broaden the application scopes of noncontact RR detection technique based on cameras.