Tracking with simultaneous camera motion subtraction by level set spatio-temporal surface evolution

The purpose of this study is to investigate a method of tracking moving objects with a moving camera. This method interprets tracking as detection of the surface generated by motion boundaries in the spatio-temporal domain and estimates simultaneously the motion induced by camera movement. The problem is formulated as a Bayesian motion-based partitioning problem in the spatio-temporal domain of the image sequence. An energy functional is derived from the Bayesian formulation. The Euler Lagrange descent equations determine simultaneously an estimate of the image motion field induced by camera motion and an estimate of the spatio-temporal motion boundary surface. The Euler-Lagrange equation corresponding to the surface is expressed as a level set partial differential equation for topology independence and numerically stable implementation. The method has a simple initialization and allows the tracking of multiple objects with non simultaneous motions. Optical velocities on motion boundaries is estimated from geometrical properties of the motion boundary surface. Several examples of experimental verification are given using synthetic and real image sequences.