TSFE: Research on Transformer and Feature Enhancement Methods for Image-Text Retrieval

In recent years, cross-modal retrieval has a wide range of applications in fields such as information retrieval and multimedia processing, where the key is to find relevant data in another modality through one modal query. In cross-modal graphic retrieval, mutual attention mechanisms are commonly used to interact image and text features. However, this interaction mechanism makes it difficult to extract separate visual and textual features for subsequent indexing steps, especially in large-scale retrieval tasks. Furthermore, in image representation, non-semantic information in image features can lead to inaccuracies in the alignment of image and text features. Therefore this paper proposes a Transformer and Feature Enhancement Method (TSFE) for Image-Text Retrieval. The method better fuses the underlying features and the higher level features by using a hierarchical Transformer coding structure on two separate branches, image and text, to proofread and localise the final output features. The Feature Enhanced Attention (FEA) module is designed in order to infer non-semantic information in the image and enhance the features of salient regions. Channel-space attention and max pooling is introduced on top of Transformer encoder self-attention to filter redundant and interfering information from channel and spatial dimensions. Experiments on Flickr30k dataset and MS-COCO dataset in this paper show improved performance in cross-modal retrieval, thus validating the effectiveness and sophistication of the proposed method.