Semantic audiovisual analysis for video summarization

This paper proposes a semantic audiovisual analysis approach for video summarization. The sequence to be analyzed is first segmented into scenes according to audio similarity. Some global clues such as loudness, the ratio of unrelated shots, and the affective relationship between the scenes and the whole sequence are employed to compute the semantic scene importance. The shots in each scene are grouped based on the luminance histograms, and the semantic shot importance is then calculated using selected audio and video features. Subsequently, key frames are extracted according to the semantic frame importance computed based on certain visual features, such as attention region and motion information. This approach is effective to generate a representative video summary whilst avoiding some disadvantages of the traditional video summarization methods. Experimental results demonstrate promising performance of the proposed approach.