Fine-grained Multi-user Device-Free Gesture Tracking on Today’s Smart Speakers

Smart speakers play an important role in smart home envision. Active acoustic sensing can enable convenient gesture interaction on smart speakers to complement voice interaction in mandatory quiet scenarios and address privacy concerns. However, existing solutions did not consider the impact of the widely adopted uniform circular geometry of commercial smart speakers on gesture tracking. To fill this gap, we propose SparseTrack to achieve fine-grained multi-user device-free gesture tracking on commercial smart speakers. We cast gesture tracking to sparse recovery intuition to address signal coherence issue on uniform circular mic-array. We then synthesize wideband measurement to eliminate spatial ambiguity caused by the insufficient spatial sampling rate of today’s smart speakers in the ultrasonic frequency band. We further design a robust trace extraction approach and properly handle the impact of the doppler effect on gesture tracking. We implement SparseTrack on COTS circular mic-array and conduct extensive evaluations. The results show that our system can simultaneously track up to 4 users’ gestures with a mean tracking error of 2.66 cm.