Video entity resolution: Applying ER techniques for Smart Video Surveillance

Smart Video Surveillance (SVS) applications enhance situational awareness by allowing domain analysts to focus on the events of higher priority. This in turn leads to improved decision making, allows for better resource management, and helps to reduce information overload. SVS approaches operate by trying to extract and interpret higher “semantic” level events that occur in video. On of the key challenges of Smart Video Surveillance is that of person identification where the task is for each subject that occur in a video shot to identify the person it corresponds to. The problem of person identification is very complex in the resource constrained environments where transmission delay, bandwidth restriction, and packet loss may prevent the capture of high quality data. In this paper we connect the problem of person identification in video data with the problem of entity resolution that is common in textual data. Specifically, we show how the PI problem can be successfully resolved using a graph-based entity resolution framework called RelDC that leverages relationships among various entities for disambiguation. We apply the proposed solution to a dataset consisting of several weeks of surveillance videos. The results demonstrate the effectiveness and efficiency of our approach even with low quality video data.