A linear model for simultaneous estimation of 3D motion and depth

A novel model for simultaneous estimation of full 3D motion and 3D position in world coordinates from multi camera sequences is presented. To this end scene flow (see Vedula, S. et al., ICCV 1999, p.722-9, 1999) and disparity are estimated in a single estimation step. The key idea is to interpret sequences of 2 or more cameras as one 4D data set. In this 4D space, a given motion model, a camera model and a brightness change model are combined into a brightness change constraint equation (BCCE) combining changes due to object motion and different camera positions. The estimation of parameters in this constraint equation is demonstrated using a weighted total least squares estimator, called structure tensor. An evaluation of systematic errors and noise stability for a 5 camera setup is shown as well as results on synthetic sequences with ground truth and data acquired under controlled laboratory conditions.