Uncalibrated perspective reconstruction of deformable structures

Reconstruction of 3D structures from uncalibrated image sequences has a wealthy history. Most work has been focused on rigid objects or static scenes. This paper studies the problem of perspective reconstruction of deformable structures such as dynamic scenes from an uncalibrated image sequence. The task requires decomposing the image measurements into a composition of three factors: 3D deformable structures, rigid rotations and translations, and intrinsic camera parameters. We develop a factorization algorithm that consists of two steps. In the first step we recover the protective depths iteratively using the sub-space constraints embedded in the image measurements of the deformable structures. In the second step, we scale the image measurements by the reconstructed projective depths. We then extend the linear closed-form solution for weak-perspective reconstruction by J. Xiao, et al. (2004) to factorize the scaled measurements and simultaneously reconstruct the deformable shapes and underlying shape model, the rigid motions, and the varying camera parameters such as focal lengths. The accuracy and robustness of the proposed method is demonstrated quantitatively on synthetic data and qualitatively on real image sequences.