Controlling perceived stiffness of pinched objects using visual feedback of hand deformation

In this study, we developed a method that influences the perception of the stiffness of pinched objects by using visual feedback in which the hand image is deformed to fit the virtual object. Several studies revealed that visual feedback can control the perception of objects' stiffness to some extent. However, most of these visual feedbacks were produced by deforming virtual objects, and few studies focused on modifying the hand's image that handled objects. This is because the deforming the hand image to fit to the dented virtual object in real time is difficult, so there was the problem of the spatial inconsistency between deformed virtual objects and superimposed hand in Mixed Reality(MR) / Augmented Reality(AR) space. In our previous works, we demonstrated that our method, which distorts the image of the user's hand, enhances the cross-modal effects of shape perception. That is, our hand image deformation method can modify the perception of stiffness, thereby inducing a stronger effect and resolving the problem of spatial inconsistency. We developed a video see-through system, which changes the hand posture and deforms the virtual object to control the stiffness feeling of the pinched object, even though the user is actually pinching a physical object with a certain value of stiffness. We conducted experiments to investigate the effectiveness of the proposed system. Our results suggested that the effect of modifying the perceived object's stiffness with our method is up to 1.6 times bigger than that with only modifying the degree of dent of virtual object.