Weighted Keywords-Guided feature Extraction and Reasoning Network for Visual Grounding

In recent years, as the basis for the downstream image-text task, Visual Grounding has attracted more and more attention. Existing models treat the feature extraction process of image and text as two completely independent parts. However, due to a lack of effective interaction between text and image, especially the understanding deviation of keywords. It tends to weaken the alignment between the extracted visual features and the target. Therefore, in this paper, we propose a Weighted Keywords-Guided feature Extraction and Reasoning Network(WKG-Net) to enhance the interaction of text and image. We design a Weighted Keywords-Guided Extractor(WKG-Extractor), which can add extra keywords information as the input to guide the further extraction from general visual features to guided visual features. Then a Weighted Keywords-Guided Reasoner(WKG-Reasoner) is introduced to fuse weighted keywords, original text and visual features together to enhance the alignment between the image and text. Enhancing the understanding of keywords will be helpful for all tasks of text and image alignment. Extensive experiments show that our method has a greater performance compared with the baseline on the three mainstream datasets.