Efficient Support for Similarity Searches in DHT-Based Peer-to-Peer Systems

Distributed hash tables (DHTs) provide a scalable and robust building block for content discovery in distributed applications such as Peer-to-Peer (P2P) systems. However, the basic DHT put/get API only supports simple exact queries. In this paper, we present a DHT-based system that efficiently supports similarity queries on multidimensional datasets. Our system embeds a logical kd-tree into the DHT's identifier space to form a distributed indexing structure, the distributed kd-tree (DKDT). We avoid creating bottlenecks, which are typical in tree- based systems, by relying on fully distributed protocols for tree management and data registrations and queries. We propose tree compressing and node shrinking techniques to efficiently support applications with high dimensionality datasets. Simulation results using both synthetic and real data show the effectiveness of our system.