Netravaad: Interactive Eye Based Communication System for People With Speech Issues

People who have speech disability due to apahsia caused by stroke, head injury or brain tumor, amyotrophic lateral sclerosis, cerebral palsy, locked-in syndrome or other motor impairments can use augmentative and alternative communication (AAC) for expressing their mind. Communication based on eye movements is very common for these people. Current systems based on eye gestures or eye tracking are very expensive, require re-calibration very often, slower in operation and might need a human interpreter in many cases. To overcome these limitations, we propose Netravaad, a communication system for people with speech disability to use their eyes to create signs and speak through eyes. Through the eye sign language Netravaani, the user can speak English language using eye signs. These eye signs are captured by a low-cost USB camera which are converted to a word, a phrase, a complete sentence, or even multiple sentences using the Sarani algorithm. The text thus created is displayed on the screen and converted to speech to play the audio via speaker making this interactive multi-modal interface for the benefit of the users. Predictive text feature is implemented to reduce the effort of the users in creating signs for all characters in a word. In addition, the need for human interpreters can be eliminated. Thus, the user can communicate to the caregiver, physician or relative through eyes with a feeling that is same as speaking via mouth. The tests conducted with volunteers from different age groups suggest that, on average, the system achieves an accuracy of 91%, 100%, and 93% in detecting alphabets, words, and numbers, respectively, among individuals aged 26 to 35 years. These findings underscore the robustness of the system, indicating its potential to provide significant benefits to the community of individuals with speech disabilities.