Enhancing Person Perception for Mobile Robotics by Real-Time RGB-D Person Attribute Estimation

Person attribute estimation is a task of great importance for a variety of real-world robotic applications. While the computer vision community has made impressive progress over the last decade, they often rely on the sole use of RGB images from surveillance datasets and large deep- learning models without considering real-time requirements. By contrast, mobile robotic platforms have to deal with restricted resources but are often equipped with RGB-D cameras, offering complementary modalities. This paper presents an approach to robustly estimate soft-biometric attributes from full-body RGB-D appearances of persons in the surroundings of a mobile robot. The effects of depth, RGB and RGB-D data as input are analyzed, taking into account runtime and resource requirements. On the robotic attribute dataset SRL, it is shown that the presented approach outperforms other state-of-the-art approaches by a large margin. Furthermore, real-time requirements are met when applied on an NVIDIA Jetson AGX Xavier or even on a mobile CPU only. Finally, a previous system for person detection, upper body orientation estimation, and posture classification is integrated to enable an even more comprehensive perception of persons in the surroundings of a mobile robot in one joint approach. The source code for training and application will be made publicly available on GitHub.