Unsupervised categorical shape reconstruction through manifolds

We tackle a new challenge of unsupervised categorical 3D reconstruction using images from different instances without any manually assigned key point correspondence. We take advantage of the observations that objects are generally captured from ground-level viewpoints, and that images transform smoothly between some characteristic viewpoints despite the difference among instances. To estimate 3D models of a category, we first embed the images into a manifold to cluster viewpoints into degenerate and intermediate viewpoints. Then, we select adequate triplets of images that capture similarly shaped objects by aligning manifolds of the different viewpoint clusters. To establish correspondence between the images, we prepare finer clusters from the original manifold, and obtain a common set of features that are at locations consistent with the average flow between neighboring clusters. Under the assumption of orthographical projection, camera parameters are estimated using the correspondence among views. Finally, visual hull for each image triplet is calculated using the silhouettes. Our method is capable of automatically reconstructing an approximate categorical model even without supervision.