Residual Enhanced Visual Vectors for on-device image matching

Most mobile visual search (MVS) systems query a large database stored on a server. This paper presents a new architecture for searching a large database directly on a mobile device, which has numerous benefits for network-independent, low-latency, and privacy-protected image retrieval. A key challenge for on-device MVS is storing a memory-intensive database in the limited RAM of the mobile device. We design and implement a new compact global image signature called the Residual Enhanced Visual Vector (REVV) that is optimized for the local features typically used in MVS. REVV outperforms existing compact database representations in the MVS setting and attains similar retrieval accuracy in large-scale retrieval tests as a Vocabulary Tree that uses 26× more memory. The compactness of REVV consequently enables many database images to be queried on a mobile device.