RoboReID: Audio-Visual Person Re-Identification by Social Robot

Person re-identification is a vision-based machine learning algorithm that enables the recognition of an individual within a group of people. In the context of Human-robot Interaction (HRI), robots capable of identifying specific humans can provide personalized interaction. This technique requires visual learning of facial features and verbal learning of voice features of individuals. In this paper, we train the Digiface dataset with over 1 million face images, and the Common Voice voiceprint dataset with 3188 hours of English dialogues. A person recognition module, RoboReId, has been implemented on a humanoid robot named EMAH. The module takes the interacting partner's face image and voice stream as input, which is then fed to a fine-tuned MobileFaceNets and ECAPA-TDNN neural network. During verification, the average accuracy of the faceId model was 58.98% and the voiceprint model was 72.25%. A humanoid robot, EMAH, is used for real-time validation of the approach in HRI. The human evaluations of the RoboReId experiment on EMAH report an accuracy score of 80% from 12 participants.