Multi-view 3D shape and motion recovery on the spatio-temporal curve manifold

In this paper we consider the problem of recovering the 3D motion and shape of an arbitrarily-moving, arbitrarily-shaped curve from multiple synchronized video streams acquired from distinct and known points in space. By studying the 3D motion and shape constraints provided by the input video streams, we show that (1) shape and motion recovery is equivalent to the problem of recovering the differential properties of the spatio-temporal curve manifold that describes the curve's trace in space-time, and (2) a local analytical description of this manifold can be computed directly from the spatio-temporal volumes defined by the input video streams. Our experimental results suggest that this manifold-based approach to joint shape and motion estimation yields shape estimates of higher accuracy that those obtained from stereo alone, allows accurate recovery of 3D curve motion, and provides significant robustness against image noise and camera calibration errors.