Correlation-based estimation of ego-motion and structure from motion and stereo

This paper describes a correlation-based, iterative, multi-resolution algorithm which estimates both scene structure and the motion of the camera rig through an environment from the stream(s) of incoming images. Both single-camera rigs and multiple-camera rigs can be accommodated. The use of multiple synchronized cameras results in more rapid convergence of the iterative approach. The algorithm uses a global ego-motion constraint to refine estimates of inter-frame camera rotation and translation. It uses local window-based correlation to refine the current estimate of scene structure. All analysis is performed at multiple resolutions. In order to combine, in a straightforward way, the correlation surfaces from multiple viewpoints and from multiple pixels in a support region, each pixel's correlation surface is modeled as a quadratic. This parameterization allows direct, explicit computation of incremental refinements for ego-motion and structure using linear algebra. Batches can be of arbitrary size, allowing a trade-off between accuracy and latency. Batches can also be daisy-chained for extended sequences. Results of the algorithm are shown on synthetic and real outdoor image sequences.