Parameterized image varieties: a novel approach to the analysis and synthesis of image sequences

This paper addresses the problem of characterizing the space formed by all images of a rigid set of n points observed by a weak perspective or paraperspective camera. By taking explicitly into account the Euclidean constraints associated with calibrated cameras, we show that this space is a six-dimensional variety embedded in R/sup 2n/, and parameterize it using the image positions of three reference points. This parameterization is constructed via linear least squares from point correspondences established across a sequence of images, and it is used to synthesize new pictures without any explicit three-dimensional model. Degenerate scene and camera configurations are analyzed, and experiments with real image sequences are presented.