Alfred: the Haptic Butler On-Demand Tangibles for Object Manipulation in Virtual Reality using an ETHD

We present "Alfred", a novel haptic paradigm for object manipulation in 3D immersive virtual reality (VR). It uses a robotic manipulator to move tangible objects in its workspace such that they match the pose of virtual objects to be interacted with. Users can then naturally touch, grasp and manipulate a virtual object while feeling congruent and realistic haptic feedback from the tangible proxy. The tangible proxies can detach from the robot, allowing natural and unconstrained manipulation in the 3D virtual environment (VE). When a manipulated virtual object comes into contact with the virtual environment, the robotic manipulator acts as an encounter-type haptic display (ETHD), positioning itself so as to render reaction forces of the environment onto the manipulated physical object. Here, we discuss the concept for this novel approach and present a simplified prototype using a single detachable tangible proxy supported by a UR5 industrial robot. Through illustrative use-cases in VR and a preliminary performance evaluation, we discuss implications for robot control and design of interaction techniques. We show that Alfred is adaptable to a wide range of virtual environments and interaction scenarios, making it a promising approach for haptic-enabled manipulation in VR, although system latency is a limitation that still remains to be addressed.