Perception and probabilistic anchoring for dynamic world state logging

Knowing precisely where objects are located enables a robot to perform its tasks both more efficiently and more reliably. To acquire the respective knowledge and to effectively use it as a resource, a robot has to go through the world with “open eyes”. Specifically, it has to become environment-aware by keeping track of where objects of interest are located and explicitly represent their geometrical properties. In this paper, we propose to equip robots with a perception system that passively monitors the environment using a 3D data acquisition system, identifying objects that might become the subject of future manipulation tasks. Our system encompasses a 3D semantic mapping and reconstruction pipeline and a storage and data merging unit for perceived information that provides on-demand modeling and comparison capabilities. Based on probabilistic logical models, we address the important perceptual subtask of object identity resolution, i.e. inferring which observations refer to which entities in the real world (perceptual anchoring). Our system can be used as a bootstrapping system for the generation of object-centric knowledge and can, in this way, be used as a mid-level perception system that enables activity recognition, scene recognition and high-level planning.