Shot Boundary Detection with Key Motion Estimation and Appearance Differentiation

With the rapid prevalence of video application, shot boundary detection (SBD) as a fundamental and essential technique in video analysis has aroused great attention. In this paper, to detect the boundary frames of video shots automatically we propose a joint learning framework over optical flow and color histogram for shot boundary detection. The proposed method carries out motion estimation on key points to obtain roughly video segmentation results, and simultaneously evaluate visual differentiation over spatial appearance to help derive more accurate boundary shot segmentation. By constructing differential sequences of consecutive frames, the abrupt shot and gradual shot are successfully distinguished with the variation in the dif-ferential sequences. The experimental performance demonstrate that the proposed method has a precision of 10% higher at least than that of the color histogram-based approach, and the F value is achieved about 10% higher.