Biological Motion-Based Emotion Recognition Through a Deep Learning Approach

Recognition of emotions has emerged as a significant trend in various artificial intelligence based applications and has recently been extensively explored. Facial expressions or speech signals form the basis of emotion-recognition research. In addition, there is a growing interest in investigating the utilization of body movements for emotion recognition because of their non-verbal nature and potential to complement facial expressions and speech signals. However, the use of biological motion for emotion recognition leverages the inherent expressiveness of human movement, while simplifying the computational and perceptual challenges associated with analyzing full-body movements. In this study, emotion recognitions from biological emotional motion was presented. A model integrating convolutional neural networks (CNN) and long short-term memory (LSTM) layers was proposed to predict six distinct emotions: happiness, sadness, anger, fear, disgust, and neutral. The dataset used in this study includes 2664 high-quality videos of emotional biological motion viewed from three different angles (frontal 0°, left 45°, and left 90°). The proposed model was evaluated using various evaluation metrics and exhibited outstanding results in all measurements, underscoring its enhanced ability to predict emotions. The CNN-LSTM model demonstrated superior precision, recall, and f1-score, attaining the highest accuracy of 95% in contrast to the other deep learning models within the Dalian emotional movement open-source set dataset, which was 85% for CNN and 83% for ConvLSTM in the test set. In this study, biological motion was employed to attain cutting-edge results in the field of emotion recognition tasks, highlighting its importance in various affective computing applications.