Tracking objects from multiple and moving cameras

We present a novel approach for multiviews tracking of moving objects observed by multiple, stationary or moving cameras. Video streams from stationary cameras are registered using ground plane homography obtained from known 3D ground plane information. In the more general case of heterogeneous cameras (a combination of stationary and pan-tilt-zoom cameras), video streams are registered using a ground plane homography and affine transformations compensating the camera motion. The detection of moving objects is performed by defining an adaptive background that takes into account the camera motion approximated by the affine transformation. We address the tracking problem by modeling motion and appearance of the moving objects using probabilistic models. The object's appearance is represented using multiple colors distribution model that provides an efficient description of the object invariant to 2D rigid and scaling deformations. The motion models are obtained using a Kalman Filter (KF) process that predicts the position of the moving object in 2D, as well as in 3D when the images are registered to the ground plane. The tracking is performed by the maximization of a joint probability model reflecting object's motion and appearance. The novelty of our approach consists in modeling multiple trajectories observed by the moving and stationary cameras in the same KF framework, and integrating multiple cues and camera views in a joint probability data association filter (JPDAF). The proposed approach allows deriving an accurate tracking of moving objects, an automatic camera handoff and the efficient management of partial and total occlusions. We demonstrate the performances of the system on several video sequences.