Augmented Reality Application for Improving Writing and Motoric Skills in Children With Disabilities

Gestures serve as an instinctive and familiar means of interaction, seamlessly extending our body language to convey messages in various social contexts. By performing predefined gestures in front of a camera, we can assign specific actions to each gesture, enabling computers to recognize and execute the corresponding commands. Leveraging computer vision and artificial intelligence libraries, we aim to develop contactless interactive game-based applications for children with disabilities. Moreover, immersive experiences captivate children’s attention and stimulate their curiosity, making learning more engaging and enjoyable. For children with disabilities who may face additional challenges in traditional learning environments, AR and immersive technologies can provide a motivating and empowering way to explore new concepts, develop skills, and achieve learning goals. Therefore, the developed application is overlaying additional layers and objects over the real-time video captured by the webcam. With this application, children will simultaneously enhance their motor skills while learning to write letters in the Macedonian Cyrillic alphabet. The application interface, including colors, sounds and animations, was designed having in mind special needs of disabled children. The developed application is tested and evaluated, and the results are presented in this paper.