MiKey: Human Key-points Detection Using Millimeter Wave Radar

Human key-points play a vital role in smart home, elderly-care, gaming, etc. The detection methods based on millimeter wave (mmWave) radars have attracted substantial attentions and been applied in various fields because of the contactless and no privacyinvasion characteristics. In this paper, we present a machining learning-based framework, Mikey, that utilizes the commercial frequency-modulated continuous-wave (FMCW) radar point cloud to estimate the key-points. The framework consists of three steps, including point cloud adaptive merging pre-processing, key-point detection and key-point post-processing. The point cloud adaptive merging pre-processing method not only addressed the challenges stemming from the point sparsity and specular reflection but also the individual and action differences. A point cloud set is constructed adaptively from consecutive multiple frames limited by both frame and point numbers. The detection model combines the temporal and spatial features by adopting a local feature encoding block and a global self-attention block. A key-point post-processing step is added to smooth the key-point predictions and remove the jagged edge by exploring the prediction coherence in consecutive frames. We also make up for the scarcity of mmWave radar dataset for a variety of targets and actions. A dataset with 88 participants for 8 different actions is collected. We evaluate the proposed framework using additional 5 participants and obtain an average mean absolute error smaller than 6cm, confirming the effectiveness of the proposed framework.