Image database retrieval of rotated objects by user sketch

The paper describes the authors' image retrieval system, which enables the user to search a grey-scale image database intuitively by presenting simple sketches. The database contains 120 different isolated objects (mostly hand tools) with arbitrary orientation. Each of these images is represented by a hidden Markov model (HMM) which has been modified in order to obtain rotation and scale invariance. Thus, no labeling of the content or the relation angle of the object is needed when adding new images to the database. This is particularly important when using query by sketch due to the skew which occurs naturally in human handwriting and in drawings. The retrieved images can be ranked according to the similarity with the query sketch using the output probabilities of the HMMs. Furthermore, the use of HMMs allows efficient pruning and thus a fast retrieval even with large databases. Experiments with the demonstration system showed good retrieval results with several users.