Spontaneous facial expression recognition by using feature-level fusion of visible and thermal infrared images

In this paper, we propose a spontaneous facial expression recognition method by using feature-level fusion of visible and thermal infrared facial images. Firstly, the appearance features of visible images and statistic parameters of thermal infrared difference images are extracted. Then, analysis of variance is adopted to select the optimal feature subsets from both visible and thermal ones. These selected features are combined as the input of a K-Nearest Neighbors classifier. We experimentally evaluate the effectiveness of the proposed method on USTC-NVIE database. The experimental results show that fusion of visible and thermal infrared features can improve the accuracy rate of negative expressions and reduce the discrepancy. Thus, it can improve the expression recognition performance.