UltraWrite: a Lightweight Continuous Gesture Input System with Ultrasonic Signals on COTS Devices

Due to the advantages of device ubiquity, natural interaction and privacy preservation, acoustic-based gesture input has received widespread attention. Researchers have proposed various techniques for different applications. However, the existing work has shortcomings of heavy data-collection overhead, non-continuous input, and performance degradation in crossuser scenarios. To overcome these shortcomings, we propose UltraWrite, an acoustic-based gesture input system that only needs extremely low data-collection overhead, supports continuous input, and achieves high cross-user recognition accuracy. The key idea of our solution is to synthesize training data of continuous gestures from isolated ones, build a lightweight continuous gesture recognition model based on connectionist temporal classification (CTC) mechanism, and design a novel decoupled model training strategy to improve its cross-user recognition capability. We have implemented prototype systems on commercial devices and conducted comprehensive experiments to evaluate their performance. The results show that UltraWrite achieves an average top-1 word accuracy of 99.3% and top-1 word error rate of 0.34%. In addition, we have also evaluated UltraWrite's robustness to the sensing distance, angle, background noise, and device. The results reveal that UltraWrite possesses strong robustness to these factors.