A query routing scheme based on the hierarchically indexed DHT for hybrid peer-to-peer search

Internet searching seems impossibly difficult with massive various resources available on the Internet. Efficient search strategies are the main concerns for complete and appropriate materials searching. Web search is centralized and depends on crawlers to discover new resources you want, which inevitably encounters with the central node bottleneck problem. So the current search strategy moves toward the peer-to-peer search mechanism, since such search mechanism can be entirely distributed. However, the pure peer-to-peer search is faced with the query message broadcast storm problem as well as the scalable problem. To deal with these problems, a hierarchical peer-to-peer search framework is proposed in the paper. The basic idea includes replacing the central node with a scalable peer-to-peer search sub-network to avoid the bottleneck of the centralized searching. Each node on the search sub-network is a local resource registry, which maintains the index information of some resource description files. For the purpose of obtaining the global knowledge to guide the query routing, the index and routing information distributed at each node of the search sub-network are mapped onto a distributed hash table (DHT). Meanwhile, nodes on the search sub-network are grouped according to resource scopes into a node group and thus the hierarchical characters are introduced. The corresponding query routing scheme also takes the hierarchical and distributed characters into account. The query is firstly propagated to neighboring nodes using a multiple-random-walkers method under the direction of the local registration node, then forwarded to the node group and propagated within group nodes under the instruction of partial hash tables, and finally forwarded between node groups with the supervision of global hash table. A prototype is developed on the common peer-to-peer developing platform JXTA. The testing result on a small-scale network shows the feasibility of the proposed framework.