MHMC: Real-Time Hand Motion Capture Using Millimeter-Wave Radar

Hand motion capture is essential for emerged augmented reality (AR) and virtual reality (VR) applications. We propose MHMC, a real-time hand motion capture system using millimeter-wave (mmWave) radar with the advantages of lighting independence, privacy protection and good user experience. When implementing such a system, we design a novel vertical neural network architecture to infer complex hand motion from limited resolution data of mmWave radar, a vision-based labeling method to acquire adequate labeled data, and effective loss functions to avoid abnormal output. The experimental results demonstrate that MHMC achieves similar accuracy as visionbased methods, along with fast processing speed, i.e., 7. 3ms per frame, for supporting real-time motion capture.