Detecting socially occupied spaces with depth cameras: evaluating location and body orientation as relevant social features

Studying people's movement and social interaction in indoor spaces requires detecting where and when interactions happen. So far, sensor-based systems for detecting interactions focused on identifying physically occupied spaces, i.e., spaces determined by the presence of physically measurable objects. In contrast, our goal is to detect socially occupied spaces, i.e., spaces occupied not by people but by social interactions that they support. We focus on detecting F-Formations, a representation of the space structure between two or more potentially interacting building occupants. We propose a system that uses infrared depth cameras to anonymously derivate body orientation from skeleton joints. The orientation of the shoulders and spine, together with the face orientation and the temporal information on occupants' walking trajectories, is used to calculate the body orientation from which socially occupied space is identified. In a user study evaluating the system, we collected data in 32 patterns within two distinct cases: individuals and dyads and evaluated the system's accuracy. The evaluation included the intended orientation and the socially accepted orientation. Experimental results show accuracy well above 90% for assessing the socially-relevant body orientation. Our algorithm to detect body orientation contributes to the automated detection of socially occupied spaces. The system shows potential to detect more complex social structures. This would have an impact on research areas that study group interactions within complex indoor settings.