MN-UIV: Multimodal Neural Network Enabling User Identity Verification Based on Millimeter Wave Radar

Basic life activities of the human body can be classified as either static or dynamic. Millimeter wave radars can monitor physiological signals in the static state and posture in the dynamic state. These biometrics can be used for identity recognition to address matching vital sign monitoring with the user identity information. However, existing studies are based on single-modal information for identity recognition using either static or dynamic information. This study proposes a framework called MN-UIV for enabling user identify verification using static and dynamic information via millimeter wave radar. The design method entails a combination of signal preprocessing and multimodal neural networks. First, the algorithm uses two radars: 1) Radar 1 obtains physiological signals (in the time and frequency domains of breathing and heartbeat) in the static state and 2) Radar 2 obtains the dynamic range angle image (DRAI) of the posture in the dynamic state. Multimodal information is later formed to extract features from multiple aspects. Here, the user identity information is verified using two designed multimodal neural networks [CNN+LSTM_Encoder (CLE), ResNet_Encoder (RNE)]. Finally, based on the predictive evaluation of five users, the average recognition rates of the two multimodal neural networks were 96% and 96.4%, respectively. Compared to the average recognition accuracy of single-modal neural networks, it has a maximum improvement of 45.2% and 31%, respectively. It can continuously monitor the signs and posture of the body, thereby reducing system idle time and improving time utilization.