Image-retrieval agent: integrating image content and text

The explosive growth of the Web and increased storage capacity have made multimedia ubiquitous, increasing the need to retrieve images, audio, and video. However, finding useful information by navigating the Web is difficult. Although search engines help users find Web information, most of these tools index only text and ignore image, audio, and video content. Partial solutions to this problem are the use of image caption text to characterize an image and using manually indexed key words. Another solution is content-based image retrieval, which uses image-processing and computer-vision techniques to retrieve images from a repository using attributes, such as color, texture, and form. We present an approach that combines a content-based image-retrieval technique using wavelets with a traditional text-retrieval algorithm to retrieve images from the Web. We incorporate these two methods in an agent called query by image content and its associated text (QBICAT) that creates a single index that includes features from the image and its textual description. Following the preprocessing of each media, the system creates a single index and uses a similarity metric for retrieval.