Soccer Video Event Annotation by Synchronization of Attack–Defense Clips and Match Reports With Coarse-Grained Time Information

The annotation of significant events within soccer videos is a fundamental step in content-based soccer video retrieval. In this paper, we propose a soccer video annotation approach based on semantic matching with coarse time constraints, where video events and external text information (match reports) are synchronized using their semantic correspondence in the temporal sequence. Unlike the state-of-the-art soccer video analysis methods that assume that the time of an event's occurrence is given precisely by the external text information, this paper considers the problem of annotating soccer videos using match reports with coarse-grained time information. The contributions of the proposed approach are: 1) we propose a more generalized approach that synchronizes video events with text descriptions using high-level semantics with coarse time constraints, rather than assuming that the timestamp is given exactly in the text description; 2) the detection of event boundaries is improved by attack-defense transition analysis (ADTA); 3) a robust and fast center circle detection algorithm is proposed for the classification of soccer field zones and ADTA; and 4) unlike conventional audio-based whistle detection, we propose a novel Hough-transform-based algorithm from the perspective of image processing. This allows the game start time to be detected, and further helps the synchronization of video and text events. The experimental results conducted on a large number of soccer videos validate the effectiveness of the proposed approach.