Dog Emotion Classification using Vision Transformer Models

Dog expressions encapsulate crucial information that reflects their current emotional states, providing an innovative approach to interpret their feelings. Although many papers have explored facial expression recognition (FER) in humans using artificial intelligence, similar research done on dogs is lacking. In this paper, we experiment to predict the emotional state of dogs broken down into 4 categories: angry, happy, relaxed, and sad. We use Mask-RCNN to handle preprocessing and utilize state of the art vision transformer (ViT) models such as dinov2-base-imagenetlk-1-layer and beit-base-patch16-224-pt22k-ft22k to analyze and classify canine emotions. Among the models, both vision transformer models achieved the overall highest accuracies compared to the CNN models. The results state that DINOv2 has the highest accuracy, followed by BeiT with the overall accuracy 84.68% and 84.50%, respectively. Color Jitter preprocessing boosts the accuracy approximately 1.98% higher than Gray Scaling. However, upon further analysis of individual class breakdowns, it was observed that BeiT models outperformed DINOv2 in the happy category, while ResNet surpassed DINOv2 in the relaxed category.