Immersive virtual environments for tacit knowledge transfer focusing on gestures: A workflow

This study presents a workflow for creating immersive virtual environments for tacit knowledge transfer. The main focus is on gestures, which are related to skill, performance, or physical emotion (not facial) e.g. sports, martial arts, playing instruments, acting, etc. The initial idea behind this design is to provide a virtual practice environment mainly for actors in order to learn new gestures or moves. However, this virtual environment can also be used by many other target audiences based on their needs. Sometimes, ambiguity is part of knowledge transfer and becomes more salient or critical when it comes to tacit knowledge, especially at early stages of transfer. Performance while maintaining believable gesture is a must have requirement for actors. Visual references (mainly video in absence of trainer) are commonly used by actors in order to learn specific moves or gestures. However, videos are limited to 2D screen view (even if stereoscopic or 360°) and do not provide chance of studying a freezing moment from all angles, simultaneously. Although this can be partly mimicked using multi-camera rig, it is still limited to the number of shots taken and only provides a linear frame sequence (mostly used as VFX). Immersive virtual environments not only eliminate this limitation but also provide one to one scale experience. In this study, the process of creating such environment is discussed in detail. This includes planning, concept design, selecting tools, establishing the environment, properly selecting or creating the virtual character(s), capturing the motion or using existing ones from different Mocap libraries, actor's interaction with VR equipment, user experience, etc. In addition to studying reference moves and gestures (frame by frame and from any angle), the user is able to observe his/her performance in VR. This can be achieved using motion capture cameras installed at the practice location. The captured content is later assigned to the user's virtual representative i.e. a 3d character created based on his/her physical body features for side by side analysis with the reference. This provides countless interaction possibilities that cannot be achieved in the real world. Few examples are: multiplying the reference character and freeze two or more different moments (frames) and create a walkthrough, creating an immersive timeline based on the actor's progress (also requires multiplying), assigning reference moves to the user's avatar to be compared with his/her movements by himself/herself or anyone else (different from side by side comparison with the reference character), and many others. What has been discussed above is fully illustrated and described in this paper including detailed figures. The contribution of this study can be extended to various fields from acting and sport to stop motion and creative art, as the processes presented in the paper are designed in the most affordable way, using hardware and software currently available to basic users.