A Novel Building Image Retrieval based on Improved ViT

Architectural image retrieval faces several challenges, such as the similarities between different classified buildings, the significant differences in illumination of the same building image, and the differences in building size. To solve the above problems, this paper introduces an approach to retrieve building images through an improved Vision Transformer (ViT). The RepMLP (Re-parameterized Multi-layer Perceptron) Block is integrated into the base model of ViT. This integration combines local and global features, thereby enhancing the extraction of local features. This approach aims to prevent the precision of small target retrieval from decreasing due to the neglect of local features. Additionally, the model is trained using a smooth loss function (Smooth-AP) to improve the accuracy of image retrieval. Our approach uses Vision Transformer, which yields superior performance in architectural image retrieval tasks. This study provides strong evidence for improving the accuracy and robustness of building image retrieval. Experimental results indicate that the improved ViT model is 3.7% stronger than the mAP of the original algorithm. In addition, the algorithm is 4.9% and 5.1% higher than R-l and R-5 respectively, and shows good detection performance in various scenes.