LifeLens: Deep Learning Based Application for the Visually Impaired

The swift rise in the field of Deep Learning and Computer Vision is transforming how we approach important problems. This paper aims at aiding the visually impaired with environment navigation using a novel prototype which utilizes prevalent solutions to the Image Captioning and Visual Question Answering problems. The IC model was implemented using two approaches - an encoder-decoder framework that incorporates a visual attention mechanism and a Reinforcement Learning (RL) Based Policy-Value Network. The performance of the two models were evaluated using the BLEU score metric. The visual attention model performed relatively better and attained an average score of 0.47. This model was combined with a Text-to-speech mechanism and was deployed on the Raspberry Pi thus facilitating the visually impaired with environment navigation. Further, a Long Short Term Memory (LSTM) network was implemented for Visual Question Answering applications.