Bangla Image Captioning With Bidirectional GRU & Attention Mechanism

Image captioning is a complex study subject in which natural language subtitles/captions are generated for pictures relying upon what is shown in them. The captioning models have demonstrated impressive results in recent times and models with attention mechanisms are becoming more popular as a result of improved performance. However, the use of attention strategies for Bengali captioning challenges has received little focus. Since no study on employing attention approaches using the Bangladeshi dataset has been done yet, this paper aims at building an attention-based captioning model for the Bengali language and Bangladesh’s geographic regions. With a caption-model composed of a GRU (gated recurrent network) and Bahdanau-attention, we used four CNN architectures: VGG19, InceptionV3, Xception, and ResNet50, for accurate features. The models were prepared by a public dataset named “BanglaLekhaImageCaptions” incorporating 9,154 pictures with Bangladeshi geological and cultural settings and Bengali annotations. Both qualitative and quantitative evaluations were used to assess the models. Experimental findings reveal that the models beat the existing works’ results and that when the encoder is Xception, the accuracy is adequate with a BLEU-I score of 87.18.