An iterative scheme for motion-based scene segmentation

We present an approach for dense estimation of motion and depth of a scene containing a multiple number of differently moving objects with the camera system itself being in motion. The estimates are used to segregate the image sequence into a number of independently moving objects by assigning the object hypothesis with maximum a posteriori (MAP) probability to each image point. Different to previous approaches in 3-dimensional (3D) scene analysis, we tackle this task by first simultaneously estimating motion and depth for a salient set of feature points in a recursive manner. Based on the evolving set of estimated motion profiles, the scene depth is recovered densely from spatially and temporally separated views. Given the dense depth map and the set of tracked motion estimates, the likelihood of each image point to belong to one of the distinct motion profiles can be determined and dense scene segmentation can be performed. Within our probabilistic model the expectation-maximization (EM) algorithm is used to solve the inherent missing data problem. A Markov Random Field (MRF) is used to express our expectations on spatial and temporal continuity of objects.