Smart indexes for efficient browsing of library collections

To enable efficient browsing and interactive querying of very large collections, such as those found in digital libraries, it is essential to provide users with summaries of query result sets. Smart indexes can be used to generate summary statistics, aggregated classification information, and/or aggregated content-based information for the result sets of arbitrary queries. We present the basic model of a smart index, as well as variations of smart indexes that are suitable when the size of summaries is large. An algorithm for generating summaries of the results of arbitrary queries is given, and algorithms for updating various summaries are discussed. Experimental results show that smart indexes generate summaries much more efficiently than traditional trees for all query areas greater than 1%-2% of the data space, with a relatively small additional storage overhead. Contrary to traditional trees, smart indexes in general perform better as the query area grows larger.