Emotion Recognition to Support Personalized Therapy: An Approach Based on a Hybrid Architecture of CNN and Random Forest

Emotions play an important role in the life of any human being, directly influencing communication and social interaction. In recent years, personalized medicine has stood out due to its ability to improve medical practice. Through processing patient data, it is possible to create strategies to individualize their care, from diagnosis to therapeutic practices. Focusing on customization of therapeutic practices, we believe that emotion recognition systems through facial expressions can contribute, especially for patients who have difficulties expressing emotions due to several reasons, including pathologies. Identifying wishes and desires during therapy through these systems can help the therapist assertively customize their approach. Aiming to contribute in this field, we developed a hybrid architecture for emotion recognition through facial expressions based on a pre-trained Convolutional Neural Network (CNN) used for feature extraction, and a Random Forest, responsible for classifying emotions. As novelty, our model was trained with a database created by combining four well studied databases (i.e.: FER-2013, Chicago Face Database, KDEF and Yale Face), what we called complete database. The proposed method achieved 70.52% accuracy in the training/validation stage and 82.92% in testing, despite all the variability between images from the different databases.