A visual tracking system for sports video annotation in unconstrained environments

A visual tracking system is presented in which a combination of techniques is used to obtain motion features of objects from a video sequence. Further processing of the motion features gives the spatio-temporal trajectories of the objects, that can be used as cues for annotation. The system solves problems found in tracking objects in unconstrained environments, such as in sports games, where there are multiple objects in motion, the camera performs pan, tilt and zoom movements, and there are objects other than the players in the background. Coarse segmentation is performed with multi-class statistical color models, constructed from samples of the representative colors of each team. Motion vectors are computed to find region correspondence between consecutive frames. Background elements are eliminated by using camera motion parameters, and other false matches are detected by analyzing motion pattern consistency. Finally, objects are registered by placing windows centered in each tracked region. An experimental realization was used to test the system for tracking players in a soccer game, but it could had also been used for generating annotation cues of videos from other sports as well.