Optimizing Image Captioning using Deep Learning based Object Detection

Many of today’s facilities available for the general public around the world are generally not accessible for people with disabilities. In the first place, accessibility is not even considered by many before providing the facilities. And among such people with disabilities are the ones with visual impairment. Many places like supermarkets, train stations, restaurants, etc., do not provide or have the necessary equipment to help such people interact with their surroundings. Naturally, they face difficulties in completing their purpose. The task of Image Captioning in Computer Vision helps in providing software solutions for such people. With the help of Image Captioning, these people can get to know their surroundings and interact accordingly. It does not entirely solve the problem at hand but it does help them to a certain extent. For these models to work correctly, they need to be optimised. This project aims at leveraging the state-of-the-art Object Detection models and combine them with the standard Image Captioning models to make the results more accurate.