Haptic interaction with objects in a picture based on surface normal estimation

In this paper we propose a haptic interaction system that physically represents the underlying geometry of objects displayed in a 2D picture, i.e., a digital image. To obtain the object's geometry captured in the picture, we estimate the physical transformation between the object plane and the image plane based on homographic information, which locates the face of the object that is viewed. We then calculate the rotated surface normal vector of the object's face and place it on the corresponding part in the 2D image. The purpose of this setup is to create a force that can be rendered along with the image without distorting the visual information. For example, if a user is touching or exploring a 2D image, he/she receives resistive force when touching a slanted face. We evaluated the proposed haptic rendering scheme using a set of pictures of objects with different orientations. The experimental results show that the participants reliably identified the geometric configuration by touching the object in the picture. We conclude with developed applications of the proposed algorithm.