Automatic video annotation and event detection for video surveillance

This paper describes a system that has been implemented with two main purposes in mind. First, detect predefined events in real-time based on the contents of continuous video sequences. Second, to help in the search of certain portions of recorded video. The key aspect of this paper is the separation between object extraction and event detection aspects and their bridging using a contents description standard (MPEG-7). We have divided the problem in three clearly distinct phases. First we extract the objects of interest along with a certain number of attributes (such as position, size, orientation, etc.) and create an annotation file using MPEG-7 standard; then we track this objects along the time. Initially we do what we call a raw tracking which consists in a simple pairing of objects in consecutive frames. Due to the difficulties in object extraction in real surveillance scenarios, the raw tracking data are usually quite noisy. These raw tracking data must be filtered in order to obtain some usable information. Finally, based on the clean tracking data, we may find events almost as they are happening.