Cooperative stereo: combining edge- and area-based stereo

For 3D reconstruction of objects, edge-based stereo techniques use geometric constraints of edges extracted from stereo images taken by a fixed, known camera configuration. The universal assumption made by most edge-based stereo algorithms is that the extracted edges are fixed, corresponding to the same feature in the 3D space. However, many objects in practice contain both fixed and smooth edges, known as occluding contours. Occluding contour edges are attributed to depth discontinuities and are view-dependent, resulting in poor reconstruction. We propose a method that combines reconstruction from an edge-based technique with a refinement process derived from area-based stereo. Using geometric constraints of straight-line segments, this edge-based technique extracts and reconstructs 3D line segments. Tracking the 3D segments by reprojecting them into images at new views, we acquire new matches with area-based techniques. These matches, no longer on the occluding contour, are used to refine the original 3D points.