Facial Emotions Recognition Using Vit and Transfer Learning

Facial Emotions Recognition is one of the celebrated and challenging tasks in computer vision. Different approaches (e.g., Convolutional Neural Network (CNN), Graph Convolutional Network (GCN), Vision transformer (Vit), etc.) have been proposed in the literature. In recent years, the most popular methods are based on CNNs, the fact which has increased the accuracy of FER. However, lack of large training databases has been a major issue in model training. Researchers use various deep-learning techniques to solve this issue, namely Transfer-Learning and Fine-Tuning. Moreover, Vit models have been a trend in the evolving use of computer vision tasks. Thus, to design a highly accurate FER system, we propose modeling through Transfer Learning (TL) technique, where a pre-trained Vit model is adopted through fine-tuning its last layer(s) to be compatible with the FER task. This work compares different pre-trained Vits models (Vit-B16, Vit-B32, Vit-L16, Vit-L32) and a Vit-CNN merging model in the RAF-DB dataset. Experiment results showed that combining Vit-b16 with pre-trained CNN model VGG16 is the best model, and our fine-tuning was found to improve accuracy in the RAF-DB dataset.