Image Captioning based Smart Navigation System for Visually Impaired

For visually disabled people, navigating the streets is a challenging activity. It’s almost always difficult for them to comprehend basic street signboards or the surrounding environment. The proposed model suggests a system that can help visually impaired people understand better their surroundings. This research proposes a novel implementation of smart spectacles based on Image Captioning and Optical Character Recognition (OCR) to ease the navigation process. The system consists of a camera that is embedded in spectacles, an Image Captioning module, an OCR module, and Text-To-Speech (TTS) module. The Image Captioning module aids in the detection of notice or signboards in the immediate vicinity, while the OCR module aids in the reading of the text on the notice boards. Text-To- Speech is used primarily to translate this data into a format that is open to the visually impaired, such as voice notifications. The proposed research makes navigating on the street less challenging for visually impaired people. It also helps them to comprehend the signboards and to make optimum use of the available facilities. The presented model can successfully generate captions based on the surroundings and can read the text whenever required. The model can read texts in English, Hindi, and Marathi, making it adaptable to the needs of the vernacular population.