Compound Facial Expression Recognition with Multi-Domain Fusion Expression based on Adversarial Learning

The emotion of human beings tends to be complex in real conditions, generating compound expressions in human faces. Compound expression recognition is an important challenge for the assessment of human complex emotion. The recognition system based on six basic expressions cannot meet the demand of compound expressions recognition. The recognition performance of models learned from the basic expressions is poor due to the small number of compound expression datasets with highly accurate labels and insufficient sample diversity. Making full use of domains outside of compound expressions in small sample datasets will help promote diversity. We propose the Multi-Domain Fusion Generative Adversarial Network (MDFGAN), which innovatively fuses the face domain, compound expression domain and basic expression domain to obtain rich expression generation capability and high accuracy recognition. Pairing the face domain and the contour-unrelated compound expression domain in the generator will expand the sample diversity. The contour-related compound expression domain and the basic expression domain will jointly improve the expression recognition accuracy of the discriminator. Finally, we conducted comprehensive experiments on CFEE-26, CFEE-7 and CK+. In the experiments, the results of MDFGAN improved 6.79% on UF1 and 8.5% on UAR.