Proposing a Physical Condition Detection System by using RGB and Thermal Images for Facial Features Detection and Facial Temperature Extraction

This paper presents a system for identifying and predicting the physical condition of construction workers such as tiredness, drowsiness, and body temperature using face detection technology. We describe a device that captures both thermal and RGB images of a worker’s face simultaneously by using a thermal camera for thermal images and a web camera for RGB images. While detecting faces from RGB images can be straightforward using the existing technologies, detecting faces from thermal images is more challenging. By analyzing the thermal images taken from the thermal camera, we extract the temperature of the worker’s body. To measure the facial skin temperature remotely, there is a method using thermography. Facial skin temperature is an important indicator for assessing drowsiness and mental stress. Therefore, it is essential to automatically detect the face in the thermal image and extract the facial skin temperature. Previously, many studies have been done to detect faces from thermography, but it was a thermal image that showed a person without a background. Therefore, we proposed a new method using SSD (Single Shot Multibox Detector) to detect faces from thermal images in real environments such as offices. For practical application, we evaluated the detection accuracy based on the difference in the size of the face area in the thermal image. In addition, the temperature of facial ROI was extracted using the facial feature points obtained by using AAM (Active Appearance Model). As a result, the face detection accuracy using SSD was about $\mathbf{9 0 \%}$, and the temperature value of the face RoI (Region of Interest) could be extracted only when the error in the facial feature point was minimal.