Surface-Assisted In-Air Gesture Recognition

In-air hand gestures can be effectively utilized to support immersive interactions in an online meeting. In this study, we propose a novel system to support in-air hand gestures by leveraging the vibrations introduced by the gestures. When a user makes hand gestures while putting her elbow on a table, vibrations are produced due to the gestures and travel through the table’s surface. Based on this observation, we collect gesture-driven vibrations and use them for gesture recognition. Our evaluations with real-world users demonstrate that we can precisely recognize in-air hand gestures (e.g., an average accuracy of 96.02%) using only commercial devices, such as smartphones and smartwatches.