The PHARAOH procedure execution architecture for autonomous robots or collaborative human-robot teams

As robotic systems grow in complexity and autonomy, the need for effective and intuitive operator interfaces and task programming tools becomes more acute. The operator must be able to preview, verify, adjust, and observe the robot's plans and actions, and make system-level decisions about the appropriate level of shared autonomy-both before and during a mission. Operator interfaces must be usable by non-experts, even in the presence of degraded situational awareness and environmental uncertainty. Indeed, when used in mission-critical domains such as space or other hazardous environments, these tools must engender confidence that tasks will be performed safely and effectively by providing transparency into the robot's decision-making processes. In this paper, we introduce the PHARAOH system for robot operation. PHARAOH integrates the PRIDE electronic procedure software developed by TRACLabs for NASA field operations with the affordance template visual task programming framework developed to command remote robot systems performing manipulation tasks with varying levels of autonomy. Together, these tools enable an operator to define and follow task procedures while providing access to state-of-the-art robot operation tools as necessary. This blending of human and robot capabilities in a well-defined architecture, covering levels of abstraction from teleoperation to autonomous execution, provides enormous flexibility for mission planners and operators, and represents a significant new tool for robot operation in the field.