Enhancing Contextualized GNNs for Multimodal Emotion Recognition: Improving Accuracy and Robustness

Emotion recognition from facial expressions is an important research area in the field of artificial intelligence. In this study, a novel deep-learning model is proposed for emotion recognition from facial expressions. The model uses a combination of modifications in COGMEN (Contextualized GNN-based Multimodal Emotion Recognition) to extract features from facial images and other modalities and capture temporal dependencies between frames. The model was evaluated on the IEMOCAP dataset, achieving an accuracy of 87.8%, outperforming (mostly) existing state-of-the-art methods. Furthermore, the model has a lower computational complexity compared to other methods, making it more practical for real-time applications. A high F1 score was reported for each emotion category, indicating good performance across all classes. The results demonstrate the potential method for improving emotion recognition from multiple modalities (i.e.: Audio, Visual & Text) and have implications for a wide range of applications such as affective computing, human-computer interaction, and mental health monitoring.