Selection of Region of Interest in Thermal Images for the Classification of the Human Emotions

The problem of thermal image processing for the classification of human emotions is considered. The work investigates three similarity measures. The first two are based on the statistic correlation of images, i.e., template/reference and candidate/selected images, for the selection of similar regions from a database with thermal images of human faces. The first measure (M1) is based on the Pearson correlation coefficient of two images as sequences, and the second one (M2) is the correlation coefficient extracted from the matrix of correlation of the two images. The last measure uses the structural similarity index (SSIM), alias M3. The reference image is defined by the user for three predefined emotions/states: neutral, sad, and happy. The computer-based simulations reveal the superiority of the correlation-based criterion. Both measures need some constraints to avoid the bad/false/ghost results. Two geometric constraints are proposed based on the decreasing of the searching area to exactly the face area, and – secondly – the left-right symmetry of the selected region. The supplementary constraints impose a two-step selection process: (i) selection of 5-10 image candidates; (ii) extraction based on proposed geometric constraints. The obtained results are satisfactory for the considered thermal database and the set of constraints is general, being valid for other potential similarity measures based on various time-frequency transforms, such as Gabor or wavelet transform.