Fully Automated AI-powered Contactless Cough Detection based on Pixel Value Dynamics Occurring within Facial Regions

Increased interest in non-contact evaluation of the health state has led to higher expectations for delivering automated and reliable solutions that can be conveniently used during daily activities. Although some solutions for cough detection exist, they suffer from a series of limitations. Some of them rely on gesture or body pose recognition, which might not be possible in cases of occlusions, closer camera distances or impediments that prevent users from performing such movements at all. Others focus on analyzing breath using audio recordings, which cannot be easily applied in crowded or loud spaces. Many of them utilize visible light data which is prone to changing lighting conditions and can lead to various privacy concerns. Taking these into account, we propose to make use of the temporal pixel value changes occurring within specific facial areas. Due to the use of a combination of object detection and signal classification models, our system allows for fully automated classification of breathing anomalies. The benchmark evaluation performed on the newly created thermal cough data set proved the reliability of the introduced solution (precision of cough detection equals 94%). Due to the use of a lightweight deep learning model, the proposed system also has huge practical value, as it can potentially be deployed on edge devices frequently sought out in markets such as autonomous vehicles, drones, smart home or military applications.