An Adversarial Attacks Resistance-based Approach to Emotion Recognition from Images using Facial Landmarks

Emotion recognition has become an increasingly important area of research due to the increasing number of CCTV cameras in the past few years. Deep network-based methods have made impressive progress in performing emotion recognition-based tasks, achieving high performance on many datasets and their related competitions such as the ImageNet challenge. However, deep networks are vulnerable to adversarial attacks. Due to their homogeneous representation of knowledge across all images, a small change to the input image made by an adversary might result in a large decrease in the accuracy of the algorithm. By detecting heterogeneous facial landmarks using the machine learning library Dlib we hypothesize we can build robustness to adversarial attacks. The residual neural network (ResNet) model has been used as an example of a deep learning model. While the accuracy achieved by ResNet showed a decrease of up to 22%, our proposed approach has shown strong resistance to an attack and showed only a little (< 0.3%) or no decrease when the attack is launched on the data. Furthermore, the proposed approach has shown considerably less execution time compared to the ResNet model.

I. INTRODUCTION

Emotion recognition is an important area of research that has gained attention in the last two decades [1]. Information like nonverbal signals in human to human communication can be transmitted by facial expression [2], [3]. For instance, information can be conveyed via a scowl or a smile. It is considered as a natural way of trying to understand the psychological state of people during communication [4].