An Augmentative System with Facial and Emotion Recognition for Improving Social Skills of Children with Autism Spectrum Disorders

Autism spectrum disorders (ASDs) are one of the complex, pervasive, and multifactorial neurodevelopmental conditions that affect one in 68 children. Scientific research has shown the efficiency of using technologies to improve communication and social skills of autistic children. The use of software systems, such as mobile applications and multimedia, increases the interest of autistic children to learn while playing games. This paper presents an Andriod mobile application called Ying which is augmented with facial and emotion recognition. This mobile app complements existing traditional therapeutical approaches to autistic children social interaction, such as Applied Behavior Analysis (ABA). Ying integrates various computer-assisted technologies to enhance autistic children’s social behavior and verbal communication skills. An experiment showing the efficacy of using Ying has been conducted and its results are presented in the study.