Integrating color and spatial features for content-based video retrieval

We present a novel scheme for content-based video retrieval by exploring the spatio-temporal information. A shot with significant content changes can be segmented into several subshots that are of coherent content, and a shot similarity measure for video retrieval can be computed from the similarity between corresponding subshots. To characterize the temporal content variations in one shot, we developed two descriptors: dominant color histograms (DCH) and spatial structure histograms (SSH). By fusing temporal information into color content, DCHs for a "group of frames" (GoF) are trying to capture the dominant colors with long durations, which would be the colors of the focused objects or background. SSH is a set of features extracted from color-blob maps to describe spatial information for one individual frame. Experimental results on real-world sports videos prove that our proposed approach achieves the best performance on the average recall (AR) and average normalized modified retrieval rank (ANMRR) for video shot retrievals.