Put: language-based interactive manipulation of objects

Our approach to scene generation capitalizes the expressive power of natural language by separating its aptness in specifying spatial relations from the difficulties of understanding text. We are implementing an object-placement system called Put that uses a combination of linguistic commands and direct manipulation. The system is language-based, meaning that its design and structure are guided by natural language. Our approach (inspired by research in cognitive linguistics) is to analyze the natural use of spatial relations, define a well-understood class of fundamental relationships, and gradually build a coherent and natural spatial-manipulation system. Just a few simple spatial relationships, such as in, on, and at, parameterized by a limited number of environmental variables can provide comfortable object manipulation. These natural commands can be used to quickly prototype a complex scene and constrain object placement. We believe that we have an extensible, predictable, and computationally feasible environment for object manipulation. We have focused first on spatial relationships because they are fundamental to many conceptual domains beyond object placement, including motion and time. These particular domains are very important to areas of computer graphics such as animation. Uses of spatial relationships in these areas can be quite complex. We briefly introduce the complexities of understanding spatial relations and summarize related work. Then we describe the core of the Put placement system, followed by its linguistic, procedural, and interactive interfaces. We conclude by discussing future enhancements to the system.