This work introduces a non-supervised algorithm for learning generative models for classification/recognition of human activities (specifically, pedestrian trajectories) with application to video surveillance. The proposed algorithm comprises two main features: (?) a set of low level dynamical models of the trajectories, estimated in unsupervised manner using the expectation-maximization (EM) algorithm and automatic model selection using the minimum message length (MML) criterion; (ii) a switching dynamical model described by an hidden Markov model (HMM) used to characterize the higher level activities. The hierarchical model with these two levels is herein denoted as switched dynamical hidden Markov model (SD-HMM). We illustrate the performance of the proposed technique for human activity recognition in a university campus.