Enhancing Multi-user Interaction with Multi-touch Tabletop Displays Using Hand Tracking

A rear-projection multi-touch tabletop display was augmented with hand tracking utilizing computer vision techniques. Touch detection by frustrated total internal reflection is useful for achieving interaction with tabletop displays, but the technique is not always reliable when multiple users in close proximity simultaneously interact with the display. To solve this problem, we combine touch detection and hand tracking techniques in order to allow multiple users to simultaneously interact with the display without interference. Our hope is that by considering activities occurring on and above a tabletop display, multiuser interaction will become more natural and useful, which should ultimately support collaborative work.