Integrated visual and haptic display translucent and flexible sheet

An integrated visual and haptic display using translucent and flexible sheet such as rubber is proposed. This display varies the compliance of the sheet in the normal direction by changing the bias tension of the sheet: tightly stretched sheet feels hard, and loosely stretched sheet feels soft Hence a user can feel compliance of various virtual objects when he pushes the sheet. This sheet is also used as a rear projection screen; an LCD projector projects the image on the sheet from its back. When the user's finger pushes the sheet, the position and deformation of the pushed point are measured by stereo vision. The deformation of the virtual object is calculated by FEM, and the graphical model of the deformed object is generated. The user sees both the image of the object projected on the sheet and his own finger pushing the image. As a result, the proposed visual and haptic display allows users to feel like pushing virtual soft objects with their fingers directly. It is ascertained that the developed display can generate variable compliance and simulate the compliance of a real soft object. Human subjects feel like seeing and pushing a virtual object directly.