CNN Learning Strategy for Recognizing Facial Expressions

The ability to recognize facial expressions using computer vision is a crucial task that has numerous potential applications. Although deep neural networks have achieved high performance, their use in the recognition of facial expressions is still challenging. This is because different facial expressions have varying degrees of similarities among themselves, and numerous variations cause diversity in the same facial images. In this study, we propose a novel divide-and-conquer-based learning strategy to improve the performance of facial expression recognition (FER). The face area in an image was detected using MobileNet, and a ResNet-18 model was employed as a backbone deep neural network for recognizing facial expressions. Subsequently, groups containing similar facial expressions were categorized by analyzing the confusion matrix, which represents the inference results of the trained ResNet-18 model, and these similar facial expression groups were then utilized to re-train the deep learning model. In the experiments, the proposed method was evaluated using two thermal (Tufts and RWTH) and two RGB (RAF and FER2013) datasets. The results demonstrate improved FER performance, with an accuracy of 97.75% for Tufts, 86.11% for RWTH, 90.81% for RAF, and 77.83% for FER2013. As such, the proposed method can accurately classify large amounts of facial expression data.