mmWrite: Passive Handwriting Tracking Using a Single Millimeter-Wave Radio

In the era of pervasively connected and sensed Internet of Things, many of our interactions with machines have been shifted from conventional computer keyboards and mouses to hand gestures and writing in the air. While gesture recognition and handwriting recognition have been well studied, many new methods are being investigated to enable pervasive handwriting tracking. Most of the existing handwriting tracking systems either require cameras and handheld sensors or involve dedicated hardware restricting user convenience and the scale of usage. In this article, we present mmWrite, the first high-precision passive handwriting tracking system using a single commodity millimeter-wave (mmWave) radio. Leveraging the short wavelength and large bandwidth of 60-GHz signals and the radar-like capabilities enabled by the large phased array, mmWrite transforms any flat region into an interactive writing surface that supports handwriting tracking at millimeter accuracy. MmWrite employs an end-to-end pipeline of signal processing to enhance the range and spatial resolution limited by the hardware, boost the coverage, and suppress interference from backgrounds and irrelevant objects. We implement and evaluate mmWrite on a commodity 60-GHz device. The experimental results show that mmWrite can track a finger/pen with a median error of 2.8 mm and thus can reproduce handwritten characters as small as 1 cm × 1 cm, with a coverage of up to 8 m
²
supported. With minimal infrastructure needed, mmWrite promises ubiquitous handwriting tracking for new applications in the field of human-computer interactions.