Weighted Subspace Filtering and Ranking Algorithms for Video Concept Retrieval

The rapid increase in the amount of digital data has posed significant challenges to retrieval systems, which are expected to effectively and efficiently filter the data and provide relevant results. Filtering is a technique that automatically selects features or data instances to represent the data, reduce storage costs, prune redundancy, decrease computation costs, enhance model learning performance, and dynamically deliver the media. On the other hand, ranking is a critical step to render as many relevant results as possible on the basis of some loss functions or similarity measures. This article presents a novel retrieval framework that consists of weighted subspace-based filtering and ranking components to address these challenges. One critical issue in multimedia retrieval is the difficulty of comparing semantics between low-level features (such as color and shape) and high-level concepts (such as sky and ocean). Because multiple correspondence analysis (MCA) is powerful in exploring relationships among high-dimensional categorical variables, it is attractive to apply MCA to explore relationships between feature categories and concept classes. However, existing correlation based algorithms are weak in capturing semantics through a numeric measure such as conditional entropy to guide the search of features. Most existing subspace-based algorithms, while considering semantics, mainly focus on the representations of features in new spaces or relationships among high-level concepts. Therefore, our filtering algorithm transfers the semantic correlation captured in the subspace to the feature weights to distinguish features and imbalanced data instances. Relevance of retrieval results is also crucial in multimedia, where effectiveness is measured by the ranking process. Our proposed algorithm is designed to overcome the limitation of the most commonly used ranking methods, which treat all data instances equally within one group during the training step. Our algorithm ranks data instances using dissimilarity values in the subspace toward both positive and nega tive one-class models from the learning phase. The idea behind this is that even those data instances in the same relevant group may have implicit different significances. For exam ple, the data instances closer to the relevant group's center could indicate more relevance than those that are further from the center. Meanwhile, a low-dimensional representation of the data instances could compactly characterize the structure of the data instance groups, making it possible to achieve an easy separation of relevant and irrelevant groups and effectively rank the retrieved results. To demonstrate the effectiveness of the proposed framework, we evaluated 30 high-level concepts and data sets from Trecvid 2008 and 2009.