Mixed Environment of Real and Virtual Objects for Task Training using Binocular Video See-through Display and Haptic Device

In order to solve the shortage of skilled workers, a training / support system that can present task instructions according to the work situation is being developed. To achieve this, a display that presents high-resolution stereoscopic images with the naked eye and a system that presents haptic sensations have been developed. By using this system, it is possible not only to instruct tasks with text information, but also to present virtual objects with a high sense of reality and to handle virtual objects using work tools. With this function, it is possible to construct an environment in which real and virtual objects coexist and treat both objects in the same way. It feels more realistic than an environment where everything is composed of virtual objects. This makes it possible to perform early task training even if all the parts are not yet available. Alternatively, even at the design stage, the design can be changed while checking the ease of assembling and maintaining the product. In this paper, the environment where real objects and virtual objects coexist is discussed. Then, the prototype system for realizing it is introduced, and the results of evaluation experiments in which both objects are handled by the same operation are reported.