Deep-Learning Based Classification of Engagement for Child-Robot Interaction

This work aims to describe the research and development of an engagement classification system based on emotion recognition through the use of facial expressions, to be used in robotic applications for children interacting with a NAO humanoid robot during therapy sessions. An emotion classification model based on different Convolution Neural Networks (CNN) architectures was first fed by faces provided from different public datasets to train and validate the emotion classification models. Two engagement classification methodologies are proposed: Method-1 first classifies 7-classes of emotions, using the ConvNeXt network, and then infers the levels of engagement based on the affective model developed by [1]; Method-2 employs the same classification network to directly classify the four levels of engagement. The results obtained for both engagement classification methodologies using the CAFE dataset proved their effectiveness, however Method-1 performs slightly better in terms of engagement classification accuracy. A novel child-robot activity with a NAO humanoid robot, that consists in a breathing-based relaxation exercise, is also proposed and the results of offline classification system showed to be promising, and with potential to be applied in psychology therapies in the future.