Effective and Efficient Query Processing for Video Subsequence Identification

Content-based video retrieval has been well investigated. However, despite the importance, few studies on video subsequence identification, which is to find the similar content to a short query clip from a long video sequence, have been published. This paper presents a graph transformation and matching approach to this problem, with extension to identify the occurrence of potentially different ordering, alignment or length due to content editing. With a batch query algorithm to retrieve similar frames, the mapping relationship between the query and the database video is first represented by a bipartite graph. The densely matched parts along the long sequence are then extracted, followed by a filter-and-refine search strategy to prune some irrelevant subsequences. During the filtering stage, Maximum Size Matching (MSM) is deployed for each subgraph constructed by the query and candidate subsequence to obtain a smaller set of candidates. During the refinement stage, Sub-Maximum Similarity Matching (SMSM) is devised to identify the subsequence, according to a robust video similarity model which incorporates visual content, temporal order, frame alignment and length information. The performance studies conducted on a long and diverse video recording validate our approach is promising in terms of both search accuracy and speed.