Tangible User Interface Everywhere Based on Imperceptible Structured Light

This paper introduces a tangible user interface based on projector-camera system. The interface is encoded by stripes in an imperceptible form. We make the first attempt to employ a neural network for extracting hand information from structured light images. The lightweight network of only 0.24 MB parameters infers hand existence, hand segmentation and fingertip locations. These results are utilized for decoding the imperceptible stripes. Experimental results show that the proposed network outperforms state-of-the-art methods, achieving a high IOU of hand segmentation (0.996), and fingertip localization precision (98.98%) within a 10-pixel range. The interactive system performs a touch detection precision of 98.78%, with an average fingertip localization error of 3.33 pixels. It supports real-time interaction with an average execution time is 24.87ms.