Influence of Thermal Imagery Resolution on Accuracy of Deep Learning based Face Recognition

Human-system interactions frequently require a retrieval of the key context information about the user and the environment. Image processing techniques have been widely applied in this area, providing details about recognized objects, people and actions. Considering remote diagnostics solutions, e.g. non-contact vital signs estimation and smart home monitoring systems that utilize person's identity, security is a very important factor. Thus, thermal imaging has become more and more popular, as it does not reveal features that are often used for person recognition, i.e. sharp edges, clear changes of pixel values between areas, etc. On the other hand, there are much more visible light data available for deep model training. Taking it into account, person recognition from thermography is much more challenging due to specific characteristics (blurring and smooth representation of features) and small amount of training data. Moreover, when low resolution data is used, features become even less visible, so this problem may become more difficult. This study focuses on verifying whether model trained to extract important facial embedding from RGB images can perform equally well if applied to thermal domain, without additional re-training. We also perform a set of experiments aim at evaluating the influence of resolution degradation by down-scaling images on the recognition accuracy. In addition, we present deep super-resolution (SR) model that by enhancing donw-scaled images can improve results for data acquired in scenarios that simulate real-life environment, i.e. mimicking facial expressions and performing head motions. Preliminary results proved that in such cases SR helps to increase accuracy by 6.5% for data 8 times smaller than original images. It has also been shown that it is possible to accurately recognize even 40 volunteers using only 4 images per person as a reference embedding. Thus, the initial profiles can be easily created in a real time, what is an additional advantage considering a solution setup in a new environment.

I. Introduction

Human-system interactions frequently require a retrieval of the key context information about the user and the environment. Apart from information acquired using various sensors, the context can be also provided by applying computer vision algorithms, e.g. person, objects, or actions detection and recognition [5]. The most challenging problems in solutions that use the vision context are often associated with poor lighting conditions [6] and security concerns [7]. Huang and Bian [8] addressed the illumination variations by adopting Gamma correction, Difference of Gauss filtering (DoG) and contrast equalization. Different approach proposed in [6] applied illuminance-invariant features, such as edge maps, Local Binary Patterns (LBP), Gabor wavelets, and local autocorrelation filters. It has been also shown that face recognition using the skin model represented in the HSV V color space works robustly regardless of the lighting conditions [9]. The face overlap can be further improved by using brightness control or by rejecting pixels with low channel values [10].