Data Collection and Real-Time Facial Emotion Recognition in iOS Apps with CNN-Based Models

This study analyzes and compares deep learning models, including Naïve-CNN, VGG16, EfficientNetV2, and MobileNetV2, for facial emotion recognition using the FER2013, and collected Zoom datasets are presented. The paper discusses the data collection of human subjects over Zoom to gather quality samples for the seven emotions targeted with the classifiers. Preprocessing steps are considered to enhance histogram information, brightness contrast, and augmentation of both datasets. The performance of these models was evaluated based on their accuracy, precision, recall, and F1-score. An iOS app was developed to test the trained models in real-time with YouTube videos. The Naïve-CNN and VGG16 models demonstrated the highest performance, while the EfficientNetV2 and MobileNetV2 models showed potential for further improvement during training and testing. The iOS app implementation showed the expected weak results of the trained models but observed capacities that the model provided for actual utilization on mobile devices. The work provides valuable insights into the challenges faced by deep learning CNN-based models for facial emotion recognition and suggests directions for future research.