VitaCap: A Vision Transformer-Based Framework for Image Captioning

Automatic image captioning, which involves generating textual descriptions from visual content, is a challenging and multidisciplinary task combining computer vision and natural language processing. This paper introduces VitaCap (Vision Transformer for Captioning), a transformer-based encoder-decoder architecture designed for effective image caption generation. The model integrates multiple feature extraction technique U-Net [1] for pixel features, Graph Convolutional Networks (GCNs) [2] for grid features, and Faster R-CNN [3] for region-based features providing a rich and comprehensive visual representation. These features are processed by the transformer encoder to capture complex dependencies, which are then utilized by the decoder to generate meaningful and contextually relevant captions. Experimental results demonstrate that VitaCap achieves promising performance across various image datasets, highlighting its potential as a robust solution for image captioning tasks.