Image Summarizer for the Visually Impaired Using Deep Learning

One of the leading problems for Humanity, particularly in this era where information and people are interconnected with text messages, are visual impairment, or have a learning disability. For this problem we aim to provide a solution that summarizes any input image into meaningful data i.e. we analyse the image data and identify the objects in it and thereby generating vocabulary and generate meaningful sentences accordingly. This can be achieved by the use of multilayer Convolutional Neural Network (CNN) to generate vocabulary describing the images, this includes living as well as non-living things. For this we used ResNet50. And a Long Short Term Memory to construct meaningful sentences using the generated keywords i.e. LSTM will use the information from CNN to help generate a description of the image and we call this, The Image Summarizer. We use the gTTS library to provide the audio output. The audio will be based on the user’s preferred language.