Deep Learning-Based Braille Technology for Visual and Hearing Impaired People

Communication can be challenging for those who suffer from both auditory and visual impairments. Even though many studies have been done to help those who have vision impairments, it is difficult to find a solution to this issue. This problem cannot be resolved by a singular method. This study seeks to create a combination of CNN (convolutional neural network) and LSTM (long short-term memory) network using Raspberry Pi. Braille is the traditional and most widely used method for helping people who are blind, and it is used in this system. It uses a combination of dots to represent letters, numbers, and punctuation. This can be done by using a device that can display text from a GSM-received SMS in braille format. An individual can use their mobile phone to convey a message to someone who is blind or deaf, and the message will be converted into Braille so that the recipient can understand it. Placing their palm on the Braille display device will allow the person with the disability (blind) to identify the characters. Additionally, using a sound bite hearing device gives hearing-impaired people a way to detect audio by biting on the vibrator. The hearing-impaired person can initiate a call via GSM using the call switch and listen to the audio from the other person through the sound-bite hearing system. Finally, the suggested method is compared with two different datasets and the result shows that the proposed method provides an accuracy more than 98%.