Photo-consistency based registration of an uncalibrated image pair to a 3D surface model using genetic algorithm

We consider the following data fusion problem. A 3D object with textured Lambertian surface is measured and independently photographed. A triangulated model of the object and two uncalibrated images are obtained. The goal is to precisely register the images to the model. Solving this problem is necessary for building a geometrically accurate, photorealistic model from laser-scanned 3D data and high quality images. Recently, we have proposed a novel method that generalises the photo-consistency approach by Clarkson et al. [2001] to the case of uncalibrated cameras, when both intrinsic and extrinsic parameters are unknown. This gives a user the freedom of taking the pictures by a conventional digital camera, from arbitrary positions and with varying zoom. The method is based on manual pre-registration followed by a genetic optimisation algorithm. A brief description of the pilot version of the method [Z. Janko et al. (2004)] has been given together with the results of a few initial tests. In this paper, we report on some new significant developments in this project. The critical issue of robustness against illumination changes is addressed and various colour representations and cost functions are tested and compared. Natural constraints are introduced and experimentally validated to simplify the camera model and accelerate the algorithm. Finally, we present synthetic and real data with ground truth, apply the improved method to the data and measure the quality of the results.