Self-Supervised GAN For Occluded Facial Expression Recognition

Facial occlusion is an important reason for the accuracy decline of facial emotion recognition in the wild. Nowadays almost all existing occluded facial emotion recognition methods are based on supervised learning. To solve this problem, we propose a generative adversarial network based framework in this paper to perform occluded facial expression recognition in a self-supervised way. The framework is composed of one generator G and three discriminators Do, D
1
, D
2
, the generator G is to complete occluded facial part and has an encoder-decoder architecture. Three discriminators have same network structure, the real-fake discriminator D
0
is to identify whether input image is generated, the mask discriminator D
1
is to identify whether occlusion exists in image, the emotion discriminator D
2
is a classifier which used to output emotion label. Four loss functions are used in training process, which are adversarial loss L
adv
; pixel loss L
pixel
; mask loss L
mask
and classification loss L
c
. To verify the effectiveness of this proposed method, experiments are designed on two datasets: CK+ and AffectNet, and we analyze the results of experiments, which proves that the framework this paper proposed can effectively reduce the influence of facial occlusion on emotion recognition.