Motion recognition by natural language including success and failure of tasks for co-working robot with human

Robots working with humans are expected to be more adaptive to failures of tasks and to have easier communication channels with those co-working humans. With this goal in mind, this paper presents a motion recognition framework for robots including a success/failure classification of tasks based on natural language representation. The proposed method consists of three parts: motion language model, natural language model and hierarchical task description. It realizes motion recognition by natural language using motion information of task execution including success and failures. When a sentence is given to the system, a state in the task description is evoked. Through the process, hierarchical relation between states is considered. The proposed framework was implemented and experiments confirmed that the proposed method identifies various motion types and provides natural language expressions. It was also confirmed that hierarchical relations among task representations are reflected to language recognition results.