Local versus global features for content-based image retrieval

It is now recognized in many domains that content-based image retrieval (CBIR) from a database of images cannot be carried out by using completely automated approaches. One such domain is medical radiology for which the clinically useful information in an image typically consists of gray level variations in highly localized regions of the image. Currently, it is not possible to extract these regions by automatic image segmentation techniques. To address this problem, the authors have implemented a human-in-the-loop (a physician-in-the-loop, more specifically) approach in which the human delineates the pathology bearing regions (PBR) and a set of anatomical landmarks of the image at the time the image is entered into the database. From the regions thus marked, the approach applies low-level computer vision and image processing algorithms to extract features related to the variations of gray scale, texture, shape, etc. The extracted features create an index that characterizes the image. To form an image-based query the physician first marks the PERs. The system then extracts the relevant image features, computes the distance of the query image to all image indices an the database, and retrieves then most similar images. The approach is based on the assumption that medical image characterization must contain features local to the PERs. The focus of the paper is to assess the utility of localized versus global features for the domain of HRCT images of the lung, and to evaluate the system's sensitivity to physician subjectivity in delineating the PBRs.