Adaptive annotation using a human-robot interface system PARTNER

We are developing a human-robot interface system, PARTNER, that takes into account the flexibility of the augmented reality approach. The prototype consists of a projector subsystem for information display and a real-time tracking vision subsystem to recognize the environment and the human's action. For teaching robot tasks, three kinds of interaction have been developed: virtual operational panel, interactive image panel and interactive hand pointer. The paper presents the fourth function, adaptive annotation function for teaching and assisting the human's task. Based on a state transition diagram, the system generates annotations adaptively to the situation by monitoring not only changes of the object's geometry but also the operator's action required for achieving the task. To detect changes of the environment, we have implemented structured light rangefinder capabilities using components of the prototype system. Experimental results of an unfolding task of a portable overhead projector device demonstrate the usefulness of the proposed system.