I am Part of the Robot’s Group: Evaluating Engagement and Group Membership from Egocentric Views

The evaluation of groups’ dynamics and engagement levels in Human-Robot Interaction (HRI) is crucial for enabling a robot to adapt its behavior and achieve sustained interaction. To address these tasks, various methods have been explored, but these aspects are often treated in isolation. In this work, we present a unified machine-learning approach that addresses the recognition of social group dynamics in a multiparty setting, using that information as a possible index of engagement from an egocentric perspective. We introduce an interaction grouping classifier that considers the robot as a potential group member. Our approach utilizes neural networks trained on a combination of an existing dataset and a newly created one, specifically labeled for this study. The proposed method leverages egocentric data to detect social groups and employs this information as an index of engagement, as it requires agents to be part of the same interacting group, including the robot. Experimental results demonstrate high accuracy in both engagement recognition and group detection. Additionally, tests on F-formations reveal complexities in scenarios involving the robot, underscoring the challenges in these configurations.