Semantic annotation of surveillance videos for abnormal crowd behaviour search and analysis

Monitoring videos captured by surveillance cameras is a very difficult and time consuming task. There is a need for automated analysis using computer vision methods in order to recognize abnormal human behaviors and assist authorities. On the other hand, crowd (group of people) behavior analysis is a new direction of research, which can be utilized for automatic detection of panic in crowds. Once, videos are processed using computer vision technologies, another problem is how this data is indexed for search and analysis, since cameras continuously capture videos resulting vast amounts of data. Oftentimes various techniques are used for indexing, storage and access of video surveillance information, which makes global analysis and search on this data very difficult. In this paper, we introduce a novel semantic metadata model based on multimedia standards to extract and annotate globally inter-operable data about abnormal crowd behaviors from surveillance videos. We demonstrate on UMN and PETS2009 datasets that generated semantic annotations enable detailed searching and analysis of abnormal crowd behaviors with complex queries.