OBTAIN: Observational Therapy-Assistance Neural Network for Training State Recognition

Children with autism spectrum disorder (ASD) often require long-term and high-quality intervention. An important factor affecting the quality of the intervention is therapists’ observation skill, by which therapists can opportunely adjust their intervention strategies based on children’s states. However, there is a shortage of experienced therapists and observational skill development is time-consuming for junior therapists to acquire. This motivates us to use data-driven deep learning method to build an OBservational Therapy-AssIstance Neural network (OBTAIN), which is a weakly-supervised learning framework for the therapy training states recognition. OBTAIN first represents children’s skeleton-sequence data as a large graph. Then a graph representation learning module is used to extract training state features. To learn spatial-temporal behavior features more effectively and efficiently, a novel structure-aware GCN block is designed in OBTAIN’s learning module. Finally, a MILnet and corresponding joint MIL loss are used to learn state score prediction from extracted features. Experimental results (0.824 AUC score) on DREAM dataset demonstrate our OBTAIN can effectively recognize the training states in autism intervention.