Testing for human perceptual categories in a physician-in-the-loop CBIR system for medical imagery

We address the following question: to what extent should the domain experts (in our case, physicians), be believed with regard to what they claim to see in images that allows them to recognize different types of pathology? Until recently our approach was to have a physician delineate the pathology bearing regions in the images. We then used what could be referred to as a scattershot approach to the characterization of these regions, meaning that we'd extract a very large number of features from these regions. Subsequently, we'd reduce the dimensionality of this feature space by using standard search techniques, such as the Sequential Forward Selection method. The article represents an alternative to the scattershot approach to initial feature extraction. We first describe the perceptual categories that the physicians claim to use for classifying images as belonging to different diseases. We then describe the specific low-level features that need to be extracted to determine the presence or the absence of the various perceptual categories. We subsequently show the discriminatory power of the perceptual categories by presenting retrieval results obtained when a query image is matched with the database images on the basis of the presence or the absence of the various perceptual categories.