Towards perceptual shared autonomy for robotic mobile manipulation

Reliability and availability are major concerns for autonomous systems. A personal robot has to solve complex tasks, such as loading a dishwasher or folding laundry, which are very difficult to automate robustly. In order for a robot to perform better in those applications, it needs to be capable of accepting help from a human operator. Shared autonomy is a system model based on human-robot dialogue. This work aims at bridging the gap between full human control and full autonomy for tasks in the domain of personal robotics. One of the hardest problems for personal robotic systems is perception: perceiving and inferring about objects in the robot's environment. We present a system capable of solving the perceptual inference in combination with a human, such that a human operator functions as a resource for the robot and helps to compensate for limitations of autonomy. In this paper, we show how a human-robot team can work together effectively to solve complex perception tasks. We present a system that asks a human operator to identify objects it doesn't recognize or find. In various experiments with the PR2 robot we show that this shared autonomy system performs more robustly than the robot system alone and that it is capable of tasks which are difficult to accomplish by an autonomous agent.