Action recognition and interpretation from virtual demonstrations

To properly perform tasks based on abstract instructions, autonomous robots need refined reasoning skills in order to bridge the gap between the ambiguous descriptions and the comprehensive information needed to execute the implied actions. In this article, we present an automated knowledge acquisition system from human executed tasks in virtual environments, and extend the knowledge processing system KNOWROB[1] to be capable to reason on the acquired data. We have set up two scenarios in a physics based simulator: creating a pancake, and garnishing a pizza dough. Users where asked to execute these tasks using the provided tools and ingredients. Using a data processing module we then collect the low-level data and the relevant abstract events from the performed episodes. The recorded data is then made available in a format that robots can understand, by using a symbolic layer to interconnect the two data types in a seamless way.