Normalcy Modeling Using a Dictionary of Activities Learned from Motion Imagery Tracking Data

Target tracking derived from motion imagery provides a capability to detect, recognize, and analyze activities in a manner not possible with still images. Target tracking enables automated activity analysis. In this paper, we develop methods for automatically exploiting the tracking data derived from motion imagery, or other tracking data, to detect and recognize activities, develop models of normal behavior, and detect departure from normalcy. The critical steps in our approach are to construct a syntactic representation of the track behaviors and map this representation to a small set of learned activities. We have developed methods for representing activities through syntactic analysis of the track data, by "tokenizing" the track, i.e. converting the kinematic information into strings of symbols amenable to further analysis. The syntactic analysis of target tracks is the foundation for constructing an expandable dictionary of activities. Through unsupervised learning on the tokenized track data we discovery the common activities. The probability distribution of these learned activities is the "dictionary". Newly acquired track data is compared to the dictionary to flag atypical behaviors as departures from normalcy. We demonstrate the methods with two relevant data sets: the Porto taxi data and a set of video data acquired at Draper. These data sets illustrate the flexibility and power of these methods for activity analysis.