Optimization on the manifold of multiple homographies

It has long been known that the set of homographies for several planes in two images, as well as the homologies of a pair of planes in several images, all lie in a 4-dimensional subspace. It has also been shown that enforcing these constraints improves the accuracy of the homography estimation process. In this paper we show that the constraints on such collections of homographies are actually stronger than was previously thought. We introduce a new way of characterizing the set of valid collections of homographies as well as suggest a computationally efficient optimization scheme for minimizing over this set. The proposed method, a generalization of Newton's method to manifolds, is experimentally demonstrated on a number of example scenarios with very promising results.