Deep Product Quantization for Large-Scale Image Retrieval

Product quantization (PQ) is a promising technique for big data retrieval. It generates compact codebooks by decomposing the high-dimensional feature space into the Cartesian product of the low-dimensional subspace. Due to the imbalance of the subspace variance and the excessive variance of the subspace, the existing methods based on PQ face inefficient codebook learning and quantization distortions. In this work, we propose an end-to-end PQ method based on deep learning to optimally divide the subspace and reduce the quantization distortion while learning shorter codebooks. A convolutional neural network model is trained by minimizing a well-designed loss function, which maps image data into clustering features, reducing and balancing the variance of each subspace. Compared with state-of-the-art approaches, our method significantly improves performance on the benchmark datasets.