Introducing SSBD+ Dataset with a Convolutional Pipeline for Detecting Self-Stimulatory Behaviours in Children Using Raw Videos

Conventionally, evaluation for the diagnosis of Autism spectrum disorder is done by a trained specialist through questionnaire-based formal assessments and by observation of behavioral cues under various settings to capture the early warning signs of autism. These evaluation techniques are highly subjective and their accuracy relies on the experience of the specialist. In this regard, machine learning-based methods for automated capturing of early signs of autism from the recorded videos of the children is a promising alternative. In this paper, the authors propose a novel pipelined deep learning architecture to detect certain self-stimulatory behaviors that help in the diagnosis of autism spectrum disorder (ASD). The authors also supplement their tool with an augmented version of the Self Stimulatory Behavior Dataset (SSBD) and also propose a new label in SSBD Action detection: no-class. The deep learning model with the new dataset is made freely available for easy adoption to the researchers and developers community. An overall accuracy of around 81% was achieved from the proposed pipeline model that is targeted for real-time and hands-free automated diagnosis. All of the source code, data, licenses of use, and other relevant material is made freely available in [1]. Clinical relevance- Detection of Self-Stimulatory behaviors from recorded videos forms a key step towards the development of automated and cost-effective technology for screening early diagnosis and tracking of developmental disorders from The authors thank the psychiatrists at Bubbles Center for Autism, India for providing us with annotations for the videos in the SSBD+ dataset. The authors acknowledge the support and the research grant from IBM GUP.