Image retrieval with sketches and compositions

We present an image search technique that is based on characterizing the strongest edges of an image. It allows queries in the form of rough line-sketches that outline the basic form and composition of an image. This permits the user to quickly and easily transform a mental picture of an image into a query. Characterizing images with signatures that represents their strongest edges, and comparing these to a dynamically generated signature of the user's sketch achieves this search. The edge signature is generated over multiple scales to account for the variation in detail of the user's line-sketch. Edge-coherence, a measure of the perceptual strength of an edge, based on its continuity, is used in generating the signature. Our search technique is fast enough to allow the system to respond with results at the completion of each stroke of a query sketch. The algorithm is robust and invariant to scale, rotation and translation. We also show how this edge-based search can be merged seamlessly with the more traditional color-region and shape-based searches to produce a search interface that is expressive, intuitive, simple, and works in the absence of sample images.