Multimedia analysis of video sources

Law Enforcement Agencies (LEAs) spend increasing efforts and resources on monitoring open sources, searching for suspicious behaviours and crime clues. The task of efficiently and effectively monitoring open sources is strongly linked to the capability of automatically retrieving and analyzing multimedia data. This paper presents a multimodal analytics system, created in cooperation with European LEAs. In particular it is described how the video analytics subsystem produces a workflow of multimedia data analysis processes. After a first analysis of video files, images are extracted in order to perform image comparison, classification and face recognition. In addition, audio content is extracted to perform speaker recognition and multilingual analysis of text transcripts. The integration of multimedia analysis results allows LEAs to extract pertinent knowledge from the gathered information.