Content-based indexing performance: size normalized precision, recall, generality evaluation

Progress in content-based image retrieval (CBIR) is hampered by the lack of good evaluation practice and test-benches. In this paper, we raise the awareness of all the parameters that define a content-based indexing and retrieval method. Extensive ground-truth, 15,324 hand-checked image queries, was developed for a portrait database of gray-level images and their backside studio logo's. Our aim was to clearly demonstrate the diminishing effect of a growing embedding on performance figures, and the establishment of a reliable ranking of several suggested CBIR gray-level indexing methods. This evaluation scheme was used first to optimize a number of parameters defining the detailed workings of each method. The database, standard image queries, ground-truth, and evaluation scripts are offered for inclusion in an evaluation site like Benchathlon.