Learning Alphabets and Numbers from Hand Motion Trajectories for Visually Impaired Children

Children with visual impairments face disproportionate challenges in learning and communicating with other kids. Integrating the learning with computer games will motivate and enhance them to learn alphabets and numbers. In this paper, we develop a computer vision-based system to recognize alphabets and numbers from hand motion trajectories captured by an RGB-D camera. While the user draws a number or a letter in front of the camera, the computer captures the hand motion trajectory and then automatic recognize it. An audio feedback is output to the blind user. Our proposed method is evaluated on a dataset contains 144 videos of 10 numbers (0-9) and 26 letters (A-Z), and achieves 86% accuracy for subject independent test.