Group Emotion Recognition in the Wild using Pose Estimation and LSTM Neural Networks

Emotion recognition in the wild determines the individual’s or a group’s emotions within non-laboratory conditions. The accuracy of these systems is relatively low due to the various conditions that can occur in real-world scenarios. A method for increasing the accuracy of detecting a group’s emotions in the wild is proposed in this paper. The approach focuses on deep learning methods using pose estimation as features of a captured video which is then passed to a Long Short Term Memory (LSTM) network for classification. In the wild, systems need to accommodate different illuminations, occlusions, blurring, and orientations as these systems will be used in real-world scenarios. Due to these accommodations, these systems’ accuracy is relatively low compared to the systems used in controlled environments. The proposed system aims to increase this accuracy by focusing on the people’s body language within the video to identify the group’s emotions. The system achieved an accuracy, precision, recall and F1 score of 91% on the VGAF dataset. The metrics derived from the results show that the proposed system beats the current state of the art and is a valid method to detect group emotions in the wild.