A Lightweight Deep Learning Based Image Captioning Model for Hindi Caption Generation

Image Caption Generation is the interdisciplinary task which merges the perspectives of Computer vision and natural language to generate descriptions for the given images. So far, Image captioning in Hindi can be said to be in its nascent phases in comparison to English. The primary difficulties are minimal sets of data and the language is too hard. In order to target the problem of the small dataset size for image caption generation, we first transliterated the dataset of Flickr30k to Hindi. In this work, we present a lightweight deep-learning architecture optimized for automatic caption generation in Hindi. The architecture works on the premise of convolutional neural networks (CNN) for image representation learning and gated recurrent unit (GRU) for encoding these features into structured sentences in Hindi. We assess the performance of the model based on standardized answers metrics such as BLEU and discuss the performance in comparison to some already existing approaches. Considering the fact that deep learning techniques are already revolutionizing accurate and fluent Hindi caption generation even for lower resource languages, this puts forth great promise of the technology. To begin with, the system has the potential to provide aid for visually impaired Hindi speakers, provides the ability to describe items on social networking platforms, and or even provide seamless captioning for educational videos.