Hearing Beyond Sight: Image Captioning and Translation with Speech Synthesis for the Visually Impaired

The ability to receive and comprehend visual content is crucial for many aspects of human existence, including education, entertainment, and communication. However, millions of people worldwide experience some form of impairment that restricts their ability to access visual data. This research outlines the following objectives: generating detailed image descriptions using a Multi-Modal caption generation system and Optical Character Recognition (OCR), translating these descriptions into low-resourced and morphologically complex Indian languages, and then converting them into speech using text-to-speech technology. This approach caters to users who are visually challenged. This paper provide a comprehensive analysis of the datasets used, the models proposed, the metrics analyzed, and the issues related to image captioning and translation that have been explored. This model aims to enable people with visual impairments to understand visual content efficiently and in their own local language, rather than relying on English. In conclusion, this paper identify potential future directions and unresolved issues in the fields of image captioning and translation.