Semantic-aware framework for Mobile Image Search

Social and Mobile are the two very characterizing trends of the Internet. Subsequently, the volume of photos with rich social, textual and contextual information increases exponentially either on mobile devices or social networks. Performing an efficient and effective mobile Image Search over social photo collection is therefore a crucial challenge. Indeed, capture the complex connections among social photos is as important as speeding up similarity search at large scale. This paper present a generic Mobile Image Search framework with hypergraph hashing. On the mobile side, users are enabled to formulate whether visual, textual or vocal queries. On the server side, we start by modeling complex connections that may exist among photos and social features using an hypergraph. To accelerate the nearest neighbor search over the hypergraph, a spectral hashing is performed. Namely, each hypergraph vertex is mapped to a binary string without loss of similarity. For unseen items in the hypergraph, a query-adaptive supervised learning is carried out to learn binary strings based on the query type. We report the initial results over NUS-WIDE collection which show that the proposed framework is promising in the field of Mobile Image Search.