Parallax-tolerant image stitching based on mesh optimization

Image stitching with large parallax has long been an important and challenging issue in computer graph and vision. Most studies focused on finding the best-fitting homography to align and seam the input images. Images with large parallax can not be mapped over the whole overlapping regions or aircrafts like ghosting and distortion may occur. In this paper, We combine the global 2D homography method with local warping based on mesh optimization. Instead of finding the best-fitting mapping relationships among input images, we align them loosely by SIFT feature points, and then refine the local areas with mesh optimization. Four constraints are considered during local warping, including the matching error, regular error, scale error and linearity error. Finally, for the image sequence, a tree-structure stitching method is proposed to reduce the accumulated error during the continuous aligning and seaming. The comparison experiments show that, our parallax-tolerant image stitching based on mesh optimization can handle parallax well.