Multi-modal topic unit segmentation in videos using conditional random fields

In this paper a novel approach of video segmentation into topic units is presented. This approach is built upon the design in which topic unit segmentation is transformed into label identification problem by defining four types of shots that reveal semantic structure of it. To implement our algorithm, four middle-level features including shot difference signal, scene transition graph, shot theme and audio type are extracted to depict the label properties of each shot, and then CRFs model is employed to identify the labels sequence. CRFs model integrates context information, so it produces accurate results in topic unit segmentation. The proposed approach is verified by two types of data: documentary and news. Experiments on testing data set yield average 86% F-measure, which illustrates that the proposed method can accurately detect most topic units in different genres of programs.