ViTH-RFG: Vision Transformer Hashing With Residual Fuzzy Generation for Targeted Attack in Medical Image Retrieval

The rapid advancement of medical technology has led to an exponential increase in the volume of medical images. To optimize clinical practice, physicians often require efficient retrieval of images from various medical image databases. However, the inherent structural and textural similarities between medical images add complexity to retrieval. Traditional methods and deep learning-based hashing techniques have limitations, including their sensitivity to adversarial examples and insufficient robustness. Adversarial learning through generative adversarial networks offers an effective solution to address these challenges. In this article, we propose a vision transformer hashing with residual fuzzy generation (ViTH-RFG) for targeted attacks in medical image retrieval. ViTH-RFG comprises a ViTH model, a prototype network, a residual fuzzy generator, and a discriminator. ViTH employs the vision transformer as a feature extraction module and, to better adapt to medical images, cleverly integrates a spatial pyramid pooling layer into the multihead attention mechanism to extract and fuse medical image features at different scales. The prototype network takes image labels as input, transmitting the extracted semantic features to the residual fuzzy generator to facilitate the generation of adversarial examples. In the residual fuzzy generator, we introduce residual fuzzy blocks and embed a fuzzy layer to handle the complex relationships between feature maps and the corresponding generated examples, thereby enhancing the model's performance in the face of targeted attacks. The discriminator is responsible for identifying whether an image is an original or a generated example. Finally, experiments on ChestX-ray14 and ISIC 2018 verified the effectiveness of ViTH-RFG in targeted attack scenarios.