Dynamic Multimodal Fusion in Video Search
Lexing Xie
Natsev, A.
Tesic, J.
IBM, Hawthorne;
This paper appears in: Multimedia and Expo, 2007 IEEE International Conference on
Publication Date: 2-5 July 2007
On page(s): 1499-1502
Location: Beijing,
ISBN: 1-4244-1016-9
INSPEC Accession Number: 9811368
Digital Object Identifier: 10.1109/ICME.2007.4284946
Current Version Published: 2007-08-08
Abstract
We propose effective multimodal fusion strategies for video search. Multimodal search is a widely applicable information-retrieval problem, and fusion strategies are essential to the system in order to utilize all available retrieval experts and to boost the performance. Prior work has focused on hard-and soft-modeling of query classes and learning weights for each class, while the class partition is either manually defined or learned from data but still insensitive to the testing query. We propose a query-dependent fusion strategy that dynamically generates a class among the training queries that are closest to the testing query, based on light-weight query features defined on the outcome of semantic analysis on the query text. A set of optimal weights are then learned on the dynamic class, which aims to model both the co-occurring query features and unusual test queries. Used in conjunction with the rest of our multimodal retrieval system, dynamic query classes performs favorably with hard and soft query classes, and the system performance improves upon the best automatic search run of TRECVID05 and TRECVID06 by 34% and 8%, respectively.
Index
Terms
Available to subscribers and IEEE members.
References
Available to subscribers and IEEE members.
Citing Documents
Available to subscribers and IEEE members.
Cart
