Facial Expression Image based Emotion Detection using Convolutional Neural Network

Person's emotions can be observed from their facial expressions. When a person feels certain emotions, his/her facial expression will usually change, such as wrinkles on the forehead, blinking of the eyes, or changes in the color of the facial skin. This paper describes how to use the Convolutional Neural Network (CNN) to identify a person's emotions based on facial expressions observed in facial images. CNN is part of deep learning method that is widely used to analyze information from images and often gives good results. Three different datasets, Facial Expression Recognition 2013 (FER2013), Cohn-Kanade Dataset (CK+), and Karolinska Directed Emotional Faces (KDEF), were used to build and test the CNN model. Such datasets contain some grayscale images in jpeg format and has been labeled with seven emotion classes (Anger, Disgust, Fear, Happy, Sad, Surprise, and Neutral). In order to detect the seven classes of emotions, the CNN architecture was built using four convolution layers and two fully connected layers. During testing, the highest performance was achieved by the KDEF dataset with an Accuracy of 0.82, Precision 0.84, Recall 0.82, and F1-score 0.81. The most easily recognizable emotion classes are Disgust and Happy, while the emotion class that is most difficult to recognize is Sad.