Learning Object and State Models for AR Task Guidance

We present a method for automatically learning object and state models, which can be used for recognition in an augmented reality task guidance system. We assume that the task involves objects whose appearance is fairly consistent, but the background may vary. The novelty of our approach is that the system can be automatically constructed from examples of experts performing the task. As a result, the system can be easily adapted to new tasks. The approach makes use of the fact that the key features of the object are consistently present in multiple viewing instances; whereas features from the background or irrelevant objects are not consistently present. Using information theory, we automatically identify the features that can best discriminate between object states. In evaluations, our prototype successfully recognized object states in all trials.