Biomedical CBIR using “bag of keypoints” in a modified inverted index

This paper presents a “bag of keypoints” based medical image retrieval approach to cope with a large variety of visually different instances under the same category or modality. Keypoint similarities in the codebook are computed using a quadratic similarity measure. The codebook is implemented using a topology preserving Self Organizing Map (SOM) which represents images as sparse feature vectors and an inverted index is created on top of this to facilitate efficient retrieval. In addition, to increase the retrieval effectiveness, query expansion is performed by exploiting the similarities between the keypoints based on analyzing the local neighborhood structure of the SOM generated codebook. The search is thus query-specific and restricted to a sub-space spanned only by the original and expanded keypoints of the query images. A systematic evaluation of retrieval results on a biomedical image collection of 5000 biomedical images of different modalities, body parts, and orientations shows a halving in computation time (efficiency) and 10% to 15% improvement in precision at each recall level (effectiveness) when compared to individual color, texture, edge-related features.