Automatic Alignment of Color Imagery onto 3D Laser Radar Data

We present an algorithm for the automatic fusion of city-sized, 2D color imagery to 3D laser radar imagery collected from distinct airborne platforms at different times. Our approach is to derive pseudo-intensity images from ladar imagery and to align these with color imagery using conventional 2D registration algorithms. To construct a pseudo-intensity image, the algorithm uses the color imagery's time of day and location to predict shadows in the 3D image, then determines ambient and sun lighting conditions by histogram matching the 3D-derived shadowed and non-shadowed regions to their 2D counterparts. A projection matrix is computed to bring the pseudo- image into 2D image coordinates, resulting in an initial alignment of the imagery to within 200 meters. Finally, the 2D intensity image and 3D generated pseudo-intensity image are registered using a modified normalized correlation algorithm to solve for rotation, translation, scale and lens distortion, resulting in a fused data set that is aligned to within 1 meter. Applications of the presented work include the areas of augmented reality and scene interpretation for persistent surveillance in heavily cluttered and occluded environments.