Data Augmentation Using Conditional GANs for Facial Emotion Recognition

In the area of computer vision, deep learning has shown better results than traditional methods. But training a deep network requires a large amount of data and it is often very expensive to collect a huge high-quality labeled dataset, especially in real applications. This is the case in the task of facial emotion recognition, where there are not many examples for training and the distribution of emotion examples can be quite imbalance. In this paper, we use conditional generative adversarial networks to augment our training set. To improve the quality of generated images, we use deep convolution network as the body architecture of generator and discriminator. Batchnorm layer is added after every convolution layer. LeakyReLU activation function is used for all layers except for the final layer of generator, which uses Tanh activation function to squeeze the output to a range between −1 and 1. We also apply Wasserstein distance as the loss function to improve the stability of our network. Using our designed conditional GANs, we first increase the number of samples in the minority classes to handle the problem of imbalanced data. Then we increase the number of samples in all classes to extend the whole training set to solve the problem of inadequate data for training. We then test our conditional GANs on a public facial emotion dataset FER2013. Experiments with and without GANs augmentation are presented. Results show that we can gain accuracy increase (4% absolute and 7% relative) after we adopt the augmentation pipelines, indicating the usefulness of our proposed methods.