Haptic Interaction with a Polygon Mesh Reconstructed from Images

Multi-view reconstruction methods are able to produce polygon meshes of a complex scene with millions of triangles which are well suited for visualization purposes. However, these large-scale, dense meshes normally cannot be haptically rendered directly with readily available APIs. In this paper, we present a method to extend meshes reconstructed from images to visual-haptic applications by using images for visual display while an improved hybrid collision detection method is used to meet the real-time requirements for haptic rendering. Moreover, three main imperfections (holes, outliers and degenerated facets) inherent in reconstructed models are also handled to help provide a smooth and consistent force feedback. Given a reconstructed model and the corresponding image, the proposed method provides a way to virtually explore the scene in the image, making it possible to appreciate art works and historic monuments within a touching distance.