Commentary Paper on “Learning and Classification of Trajectories in Dynamic Scenes: A General Framework for Live Video Analysis”

The paper describes a general platform for live video analysis. The first stage of the platform is to build a topological scene description by learning the location of nodes (i.e. zones), which are called points of interest. There are two kinds of points of interest, the entry-exit zones (areas where moving object appear and disappear in the scene) and the stopping zones (areas where the moving objects have slow speed or remain in a circle of radius R for more than t seconds). The zones are modelled by 2D Gaussian methods. The routes between nodes are learned considering only the spatial location of trajectories in the image scene and using fuzzy C means (FCM) clusterization of the trajectories that begin in an entry zone, end in an exit zone and do not remain in a stop zone. The main trajectory cluster points are aligned using dynamic time warping and merged if the Euclidean distance is lower than a threshold.The second stage consists in the modelling of the paths by introducing not only the spatial location of the trajectories but the dynamics as well to analyze behaviour. The spatio-temporal path properties are encoded using Hidden Markov Models. The platform makes one model for each cluster computed in the previous stage. The training of each HMM model is done with the paths associated to each FCM cluster. The platform adds new models by using a batch update procedure. Trajectories that do not fit in any of the models are collected and re-clustered periodically. The HMMs are updated using maximum likehood linear regression (MLLR). Each time a new trajectory is classified into a path, a transformation is learned and applied to the mean of each of the HMM states updating its corresponding path model.The last stage comprises the behaviour analysis. Each novel trajectory detected is classified into a path by comparison with all the HMMs using forward-backward procedure finding the HMM with the maximum likelihood. Anomalous trajectories are recognized deciding that its likelihood is low, by comparing the likelihood with a decision threshold. The decision threshold is learned during training. The platform also provides an online tracking analysis method. In this case a small window of the last trajectory points is analysed, this window is constantly updated with incoming points. The live tracking classification is done considering only the most recent points of the window by comparing the points with the HMMs to estimate the likelihood at each time the window is updated. The platform can detect abnormalities during live tracking by a similar method used with complete trajectories. Path prediction is described using the HMMs by calculating the top 3 best fit paths determined with the HMM likelihoods and then estimating the probability of the incomplete trajectory to remain in one of those paths.