Integrating Different Levels of Automation: Lessons From Winning the Amazon Robotics Challenge 2016

This paper describes Team Delft's robot winning the Amazon Robotics Challenge 2016. The competition involves automating pick and place operations in semistructured environments, specifically the shelves in an Amazon warehouse. Team Delft's entry demonstrated that the current robot technology can already address most of the challenges in product handling: object recognition, grasping, motion, or task planning; under broad yet bounded conditions. The system combines an industrial robot arm, 3-D cameras and a custom gripper. The robot's software is based on the robot operating system to implement solutions based on deep learning and other state-of-the-art artificial intelligence techniques, and to integrate them with off-the-shelf components. From the experience developing the robotic system, it was concluded that: 1) the specific task conditions should guide the selection of the solution for each capability required; 2) understanding the characteristics of the individual solutions and the assumptions they embed is critical to integrate a performing system from them; and 3) this characterization can be based on “levels of robot automation.” This paper proposes automation levels based on the usage of information at design or runtime to drive the robot's behavior, and uses them to discuss Team Delft's design solution and the lessons learned from this robot development experience.