Automatic pathological gait recognition by a mobile robot using ultrawideband-based localization and a depth camera

Human gaits constitute significant health information because they indicate body function conditions, such as sensory, motor, and cognitive functions. Monitoring gait patterns helps to find weakened body parts and to prevent diseases before they become serious. In this paper, we propose a novel method to monitor human gaits in indoor environments by using a mobile robot and artificial intelligence. A mobile robot periodically collects 3D skeleton data by using an installed depth camera, and the collected data are used to extract gait parameters and to classify normal, antalgic, stiff-legged, steppage, lurching, and Trendelenburg gaits via the proposed system. Ultrawideband (UWB)-based localization with Kalman filtering and odometry-based state estimation is adopted to allow a mobile robot to rapidly and accurately reach the target human. We use both 3D skeletons and joint-based angle data to achieve improved pathological gait classification performance. A bidirectional gated recurrent unit (Bi-GRU) architecture is adopted for fusion at the feature level. An accuracy of 97.64% is achieved by using the proposed multiple-input model, whereas 96.63% and 94.17% accuracies are achieved by the skeleton-and joint-based angle input models, respectively. The proposed automatic gait analysis method can contribute to improving untact smart home care and quality of life.