3D reconstruction from co-registered orthographic and perspective images: theoretical framework and applications

In this paper a complete framework for 3D reconstruction is proposed which is based on co-registered technical drawings and images. Instead of gaining 3D information from multiple perspective images of a scene, we use only a single perspective image which is augmented by a registered 2D drawing of the scene. By superimposing images with drawings a new type of document is created which we call co-registered orthographic and perspective (COP) image. A COP image contains two views, whose spatial relationship in regard to each other is known. The corresponding features within the COP image, i.e. between the perspective view and the orthographic view, provides the same information as if two separate images were used. However, correspondence identification becomes much simpler and less erroneous, since the two views are overlaid. We provide a complete framework for registration of calibrated images with drawings as well as methods for reconstruction of primitive objects in industrial environment. Finally, results on scenes in industrial environment are presented to demonstrate the feasibility of the proposed method.