Hierarchically multi-modal indexing of soccer video

In this paper, we propose a novel mechanism of hierarchically indexing soccer video using visual, audio and textual cues. Firstly, video is indexed with information from pure video and audio respectively. Then, video is segmented into physical shots based on visual features, and then identified as syntactic shots according to broadcasting rules. Audio is analyzed to get physical contents and then syntactic contents. Finally, fusing the informative clues from video, audio and text extracted from video, we get semantics of soccer video and the shots are annotated as semantic shots. Our experiment shows that this mechanism is effective and robust