Multimodal Deep Learning Framework for Non-Intrusive Autism Spectrum Disorder Detection

The non-intrusive detection of Autism Spectrum Disorder (ASD) marks a significant advancement in early diagnosis and intervention. This approach allows users to upload videos to a web interface, where visual and auditory data are processed separately. Visual data is analyzed using a customized VGG19 Convolutional Neural Network (CNN) to identify facial expressions and movements indicative of autism, while auditory data is evaluated by a Recurrent Neural Network (RNN) to detect auditory cues like tone and rhythm. A voting mechanism across 10 frames determines the visual classification, labelling the subject autistic if most frames indicate autistic traits. Audio analysis provides an independent prediction. The subject is classified as autistic if either analysis indicates autistic features. By leveraging deep neural networks to process multimodal data, this method offers an accessible, early ASD detection tool for caregivers and professionals, promoting timely interventions and improved outcomes.