Grounding Verbs for Tool-Dependent, Sensor-Based Robot Tasks

Speech-based robot instruction is a promising field in private households and in small and medium-sized enterprises. It facilitates the use of robot systems for experts as well as non-experts, even while the user executes other tasks. The common approach is to map action verbs to robot tasks represented by well-defined interfaces. While this method works well for a wide variety of robot motions, it contains the necessity for defining additional robot tasks for each change in the physical properties of the manipulated objects. To overcome this drawback, we contribute a novel approach for defining complex and sensor-based robot tasks called Combined Verbalized Effects. It allows to systematically ground action verbs to sensor-based robot motions, while considering accessible tools and highlighting the motion primitive combinations complex motions are made of. Furthermore, real-world examples and a user study show the applicability of our approach.