Teaching agents to learn: from user study to implementation

In order to understand how best to create a system that can learn, we conducted an experiment in which users attempt to train an intelligent agent to edit a bibliography. In our experiment, a researcher sat behind a curtain masquerading as the intelligent agent, which we named Turvy. To make the experiment as useful as possible for constructing a real software agent, we set limits on Turvy's abilities. In particular, we limited the types of instructions Turvy could understand, the features of text it could observe, and the generalizations it could learn. In the course of the experiment, Turvy had to do things such as select many short strings of text, keep track of errant white space, and handle names containing "van" or "de." Users invented their own teaching methods and spoken commands to help Turvy learn even the most difficult elements of the experimental tasks. Armed with the results of these experiments, we implemented an interactive machine learning system, which we call the Configurable Instructible Machine Architecture (Cima). Designed to acquire behavior concepts from few examples, Cima keeps users informed and allows them to influence the course of learning. We have tested Cima on transcripts from the Turvy experiment and found that it learns at roughly the same level as the researchers masquerading as Turvy.