Refining Expression Recognition in Pixel-Based Emotion Detection Using Hybrid Deep Learning

Emotion recognition is an important prerequisite for human-computer interaction, virtual reality, and surveillance technology. This study proposes a new deep learning model with CNN, LSTM and Transformer topologies to increase the dependability of the emotion recognition analysis based on face photos. The model underwent evaluation using two prevalent datasets: FER2013 and AffectNet, which are two such large-scale publicly available databases. That is why, in the case of the hybrid model, the achieved accuracy was 74.6% on FER2013, which is an improvement over baseline CNN by 5 percentage points. On AffectNet, it achieved an accuracy of up to 78.2% and a macro F1 score of 72.9%, and this result is significantly higher than exclusive CNN and LSTMs by about eight percent accuracy improvement. The introduced transformer module enabled the classification of long-range dependencies and contextual relations in facial expressions, thus reducing the misclassification of fine-grained categories of fear, surprise, disgust, and contempt. When comparing the results of the hybrid model with existing models such as VGGFace and ResNet, the researchers observed that the hybrid model outperformed them on both datasets. Furthermore, Grad-CAM visualization revealed that the model focused on some of the crucial parts of the face, such as the eyes and lips, and thus improved its robustness. The hybrid model suggestion to integrate spatial, temporal, and contextual features posits the work as capable of informing enhancements in emotion identification systems with applications in real-time systems, virtual worlds, and human-computer interfaces. The results imply that the hybrid model could be a proper way to improve face emotion recognition or deep learnings.