Smart query expansion scheme for CDVS based on illumination and key features

Given a query image, retrieving images depicting the same object in a large scale database is becoming an urgent and challenging task. Recently, Compact Description for Visual Search (CDVS) is drafted by the ISO/IEC Moving Pictures Experts Group (MPEG) to support image retrieval applications, and it has been published as an international standard. Unfortunately, with regard to applications with hugely mutative illumination, perspective and noisy background, CDVS suffers from an inevitable performance loss. In this paper, firstly we introduce the query expansion to address performance loss caused by the scene complexity in CDVS. Secondly, a query expansion instance selection method based on illumination is proposed, which achieves better performance. Thirdly, we adopt a key feature matching score based weighted strategy in basic query expansion to improve retrieval performance. We evaluate our proposed methods on the Oxford (5K images) dataset and a reality traffic vehicle dataset (12K images), and the result shows that the proposed methods boost mean average precision (MAP) by 7% ∼ 10% in Oxford dataset and 7% ∼17% in vehicle dataset.

I. Introduction

In recent years, with the popularity of digital image devices in various areas, the demand for directly visually searching based on image has become stronger and stronger. To address the computation complexity, memory load and bandwidth limitation in visual search, the MPEG drafted the Compact Description for Visual Search [1] standard, in which image compact descriptor consists of global descriptor (GD) and local descriptor (LD) generated from selected SIFT points. CDVS is proved to achieve remarkable improvements in increasing compactness of image descriptor and reducing the computation complexity and memory demand while obtaining high MAP. However, in feature points based visual searching system, a large range of scale changes, great affine transformation and quantization distortion in descriptor extraction can cause some target objects missed in retrieval. Besides, in some complex application scenes, such as traffic vehicle retrieval, the illumination may change greatly from day to night, which leads to much image retrieval performance loss because of the huge illumination difference. All these issues prevent image retrieval from practical applications.