Learning Object Spatial Relationship from Demonstration

Object spatial relationship refers to the use of predicates to describe the relative positional relationship between two objects. For robots, understanding the spatial relationships between objects in scenes can contribute more robustness to the interaction process between robots and the environment in a closed-loop way. Traditional methods typically use models to learn 2D features of images to determine spatial relationships between objects, which has certain limitations in understanding 3D spatial relationships. In this paper, first, we propose a method to learn object spatial relationships from human demonstration, which utilizes spatial probability models to characterize spatial relationships between objects. Second, in order to facilitate the addition of understanding of new spatial relationships, we use a monocular 3D object detection method to construct new probability distribution models of spatial relationships from image and language demonstrations. Finally, we use SVM model to distinguish different spatial relationships and recognize spatial relationships between objects in images. We test the proposed method using the NUSCENES dataset and tabletop scenes to confirm its efficacy. The experimental results show that our method acquire high spatial relationship recognition accuracy and has the potential to enable robot manipulation more robust.