Development and Validation of an iPad-based Serious Game for Emotion Recognition and Attention Tracking towards Early Identification of Autism

The diagnosis of Autism Spectrum Disorder (ASD) and Attention Deficit Hyperactivity Disorder (ADHD) can be challenging due to limited accessibility and subjective assessments. Autistic individuals often present difficulties in emotional regulation, emotion recognition and imitation, and in maintaining focus. Emotional expressions and attention are thus hallmarks of ASD and ADHD and can be analyzed to identify these conditions. In this study, we developed and validated a serious game that integrates emotion recognition and attention tracking as a novel tool for identification of ASD and ADHD. Leveraging the TrueDepth camera capabilities, our game provides a cost-effective and user-friendly alternative to current face-tracking technologies. We compared the accuracy of emotion recognition using Euclidean distance with calibrated reference expressions and a calibration-free system based on a machine learning model using Random Forest. We also identified children at risk of ADHD using the Bells test and constructed a machine learning model, utilizing Support Vector Machine and Leave-One-Out Cross Validation, trained on attention data and game data to predict this risk. Our game was tested on 20 adults to validate the emotion recognition system, and then on 17 children of the primary school to assess usability and test the constructed models. The emotion recognition system achieved an accuracy of 0.78 for adults and 0.45 for children, while the machine learning model predicted seven emotions in children with an accuracy of 0.50, suggesting the potential for eliminating the need for calibration. The model also obtained good results in predicting valence and arousal values. The attention model showed excellent validation scores (accuracy: 0.94), indicating the possibility of extending it to a larger cohort. The System Usability Score was excellent (85.0), and children found the game enjoyable, making it a promising tool for ASD and ADHD identification.