Minimal projective reconstruction including missing data

The minimal data necessary for projective reconstruction from image points is well-known when each object point is visible in all images. We formulate and propose solutions to a family of reconstruction problems for multiple images from minimal data, where there are missing points in some of the images. The ability to handle the minimal cases with missing data is of great theoretical and practical importance. It is unavoidable to use them to bootstrap robust estimation such as RANSAC and LMS algorithms and optimal estimation such as bundle adjustment. First, we develop a framework to parameterize the multiple view geometry needed to handle the missing data cases. Then, we present a solution to the minimal case of eight points in three images, where one different point is missing in each of the three images. We prove that there are, in general, as many as 11 solutions for this minimal case. Furthermore all minimal cases with missing data for three and four images are catalogued. Finally, we demonstrate the method on both simulated and real images and show that the algorithms presented in the paper can be used for practical problems.