A Communication Aid System for Deaf and Mute using Vibrotactile and Visual Feedback

Assistive technology is quite necessary to establish a two-way communication among deaf, mute and normal people where none of them needs to acquire knowledge of sign languages. In this paper, we propose a novel communication aid system that assists the deaf and mute to communicate independently with no use of sign language. Usually, deaf and mute, and normal people communicate with each other using visual references and simple sentences when none of them knows the sign languages. These small talks are mainly based on some contexts or keywords which can be delineated visually. Again these keywords can be classified based on syllables for the purpose of vibrotactile outputs so that the speech is easily comprehended. Therefore our proposed system is emphasized on both visual and vibrotactile feedback while communicating. An Android app has been developed which is focused on multimodal approach can be able to convert speech to visual contexts and vibrations, and similarly, the contexts and vibrations can be converted to speech. To validate the program, the system was experimented by six deaf and mute participants, two normal persons, and two examiners. Finally, the study of our experimentation reflects the effectiveness and usability of our system.