Automatic Video Annotation System for Archival Sports Video

Many of the archival sports broadcast videos do not have the luxury of rich metainfo of the sports (e.g. player statistics, informative player profile, supplementary field info, and etc.) available for enhancing the viewing experiences to the level what we see on TV now. To tackle this shortcoming, in this paper we propose an automatic annotation system for sports video, by using an assembly of computer vision techniques that fuse multi-moduality information to provide better metainfo of the archives. Specifically, shoot boundary detection is first applied to cut video into single shoot segment, a tensor based compact representation of each segment is fed to spectral clustering to differentiate the various views composed into the broadcast video. Player detection and tracking, as well as the optical character recognition (OCR) for on-screen graphics are then deployed to acquire the name and the corresponding players appeared in the segment. These info are finally fed into parser to obtain detail player info available through public database. We have demonstrated the proposed system is efficient in correctly annotating several archive baseball broadcasts with minimal human intervention.