Quick and Efficient Large Scale Approximate Nearest Neighbor Search on High-Dimensional Data

Large scale approximate nearest neighbor search on high-dimensional data is always a critical research question, including recommendation systems and image search. Graph-based ANNS algorithms have received more and more attention due to its accuracy and efficiency. Nevertheless, the graph-based ANNS algorithms faces the following challenges: computation latency and memory requirements. In this work, we present a novel graph-based similarity search algorithm called HNSWQ, which optimizes the HNSW algorithm by building quick and comprehensive uppers without using compression. Those uppers can bring high-quality entry point to accelerate query processing by reducing the number of points to be accessed during the search and shorten the whole computation latency. At the same time, we consider the parallelism of GPU and take advantage of GPU to shorten the build and search time of the novel HNSWQ algorithm. Finally, we verify the presented algorithm on several representative datasets. Compared with the state-of-the-art graph-based algorithm, our novel method can acquire great accuracy and search speed.