Head Gesture and Voice Based Learning App for Children with Autism of Nepal

This paper is focused on the development of a learning platform for children with autism of Nepal in the form of a desktop application. Autism spectrum disorder is marked by core features including impaired social communication, and restrictive and repetitive behaviors and interest (Diagnostic and Statistical Manual of Mental Disorders 5th Edition: DSM 5). The main objective of this paper is to develop a desktop-based application that will aid children with autism in learning new words and in recognizing the various objects with the help of pictures and audio in games. The front end of the application has three main sections. Two of them were based on voice and pictures; one is speech training and the other is speech game. The third section is a game that consists of a game character which makes movements as per the head and facial gestures such as left and right, and jump. The performance parameters like accuracy of the character movements and voice recognition were calculated and were obtained as 97.09% and 85% respectively. The application was tested among neurotypical as well as people with autism, with the age ranging from 4 to 22. The tests showed the positive attitude toward the application for the learning by both groups however in differing ranges. Their involvement was measured against different parameters like their learning time, scores in the game etc. Thus, it can be concluded that this kind of learning platform can help the children with autism to grow their skills in different fields. Also, with the implementation of AI analytics and cloud computing into the data collected through this application, in future, this could be an initiating step toward the industry 4.0 approach for the children with autism.