A Multilingual Handwriting Learning System for Visually Impaired People

Visually impaired people have previously been brought into learning and educational systems through various forms of assistive technology, such as haptic feedback systems. Haptic systems generally need expensive equipment and support from sighted teachers. Moreover, the learning has always been carried out with letters of different alphabets mapped into some tactile pattern. Writing is a big concern for the visually impaired as most official work, like signing, is still carried out by conventional handwriting methods. Most of the existing systems are limited to teaching a single language’s alphabet and basic grammar or may not provide feedback to let the learners know of their learning progress. Therefore, the objectives of this research are to develop an efficient system that includes voice-over guidance to teach writing in multiple alphabets to visually impaired people and to evaluate the performance of the proposed system. As such, a system was developed for teaching multilingual alphabets to visually impaired people with voice instructions. With the aid of a voice-over guide, learners were able to write letters with a stylus on a graphics pad. The progress assessment of the learners is carried out by an image processing algorithm and scored by a machine learning (ML) model. The Random Forest model was used due to its high accuracy (f1-score of 99.8% on test data) among the existing ten different ML algorithms. Finally, the performance and usability of this system were evaluated through an empirical study replicated with 16 participants, including four teachers and twelve visually impaired people. It was found that visually impaired people made fewer attempts to learn handwriting with the proposed system than with the normal handwriting teaching system. 100% of the participants agreed to recommend the system in the future.