Two-stage sparse graph construction using MinHash on MapReduce

Image graph attracts attention from researchers due to the empirical success of graph based semi-supervised learning (SSL) methods and tasks such as image clustering, image navigation. Despite its simple structure, overwhelming scale of online images makes image graph construction a difficult problem. The challenge lies in time-consuming computation, and the difficulty of storing, processing the resulted graphs of huge size. We propose a novel method of image graph construction on MapReduce for large-scale data. The method consists of two stages: the first stage separates images into overlapping groups called image pools by using hash method, and the second computes pairwise similarities for pairs of images that are grouped into common pools. Both stages are performed on MapReduce. Our experiments on large-scale data show that the proposed method generates more sparse image graphs that reserve same or improved accuracy when comparing with previous method.