Video track screening using syntactic activity-based methods

The adversary in current threat situations can no longer be identified by what they are, but by what they are doing. This has lead to a large increase in the use of video surveillance systems for security and defense applications. With the quantity of video surveillance at the disposal of organizations responsible for protecting military and civilian lives come the issues regarding storage and screening of this data. This paper defines a screening and classification method based upon activity-based screening and recognition to provide that filtering mechanism. Activity recognition from video for such applications seeks to develop semi-automated screening of video based upon the recognition of activities of interest rather than merely the presence of specific persons or vehicle classes developed for the Cold War problem of “Find the T72 Tank.” This paper examines the approach to activity recognition, consisting of heuristic, semantic, and syntactic methods, based upon tokens derived from the video as applied to relevant scenarios involving behavior as captured from entity tracks. The proposed architecture discussed herein uses a multi-level approach that divides the problem into three or more tiers of recognition, each employing different techniques according to their appropriateness to strengths at each tier using heuristics, syntactic recognition, and Hidden Markov Model's of token strings to form higher level interpretations. Performance of activity-based screening and recognition as applied to example scenarios has been demonstrated to reduce the quantity of tracks (analogous to video frames) by orders of magnitude with little loss of relevant information.