Deep Semantic Ranking Hashing Based on Self-Attention for Medical Image Retrieval

With the rapid progress of medical image technology, medical image retrieval has attracted wide attention in medical data processing fields. Deep hashing methods have been proven effective for massive medical image retrieval. However, existing medical image retrieval methods ignore lesion context and category-level semantics, so it is difficult to correctly correspond to the context information and category of the lesion, resulting in poor performance. This paper addresses this dilemma with a novel Deep Semantic Ranking Hashing Based on Self-Attention (DSHA) approach. We first divide the medical triplet into smaller patches and send them to the multi-head self-attention module, which can more effectively encode the context information of the lesion and the interaction among patches. Meanwhile, the weight-sharing triplet networks are used to learn hash codes, which are forced to be semantically aligned. The proposed semantic enhancement loss effectively enhances the category-level semantics of hash codes. In addition, we added a semantic ranking penalty loss to optimize the retrieval accuracy. Extensive experiments on diverse medical image datasets prove that our DSHA method achieves remarkable results compared with the state-of-the-art medical image retrieval methods.