Interaction with hand gesture for a back-projection wall

(Semi)immersive displays like back projection walls have become widely used in various visual domains. Despite the superior image quality of 'through-the-windows VR' users are still not satisfied with the interaction capabilities - mostly due to the extra wiring for the interaction devices. This paper represents a hand gesture interaction system for a back-projection wall environment, supporting object manipulation (translation, rotation, and uniform scale) and selection tasks through the vision-tracking technique. We propose to use thimble-shaped fingertip markers made of white printing paper with a 'black light' source. Gesture parameters are calculated by 3D positions of the marked fingertips and their pinching states.