Enriching Transformer using Fourier Transform for Image Captioning

Transformers have an encoder-decoder architecture which is widely used as an image captioning model. Identifying the key elements and the underlying relationships between them is the challenge of image captioning. The introduction of the Fourier transform in this architecture is expected to perform better than other state-of-the-art models. Fourier transform is a signal processing tool that allows us to decompose a signal into its frequency components. In signal processing, an image is often treated as a two-dimensional signal, where the brightness of each pixel represents the amplitude of the signal at that location. With this intuition, an additional Fourier layer is added to the transformer architecture to enhance its performance. The Flickr dataset is used for training the proposed model in its initial phase. The model is tested against state-of-art-model using various evaluation metrics like BLEU-n, meteor, rouge scores, etc. During the evaluation, the suggested approach produced better outcomes.