Non-sequential video content representation using temporal variation of feature vectors

An efficient and low complexity algorithm for non-sequential video content representation is proposed. Our method is based on extracting a set of limited but meaningful frames (key-frames), able to represent the video content. The temporal variation of feature vectors for all frames within a shot, which form a trajectory in a high dimensional space, is used for key-frame selection. In particular, key-frames are extracted by estimating appropriate curve points, able to characterize the feature trajectory. The magnitude of the second derivative of the frame feature vectors with respect to time is used as a curvature measure in our approach. Due to low complexity of the algorithm, the proposed method can be easily implemented in hardware devices of even low processing capabilities thus can be embedded in many consumer electronics systems. For feature vector formulation, the video is first analyzed and several descriptors are extracted using a multiscale implementation of the recursive shortest spanning tree (RSST) algorithm, which significantly reduces the segmentation complexity. In addition, the whole procedure exploits information that exists in MPEG video databases so as to achieve a faster implementation. Finally, the extracted descriptors are classified using a fuzzy formulation scheme. Experimental results to real-life video sequences are presented to indicate the good performance of the proposed algorithm.