Multistage Cable Routing Through Hierarchical Imitation Learning

We study the problem of learning to perform multistage robotic manipulation tasks, with applications to cable routing, where the robot must route a cable through a series of clips. This setting presents challenges representative of complex multistage robotic manipulation scenarios: handling deformable objects, closing the loop on visual perception, and handling extended behaviors consisting of multiple steps that must be executed successfully to complete the entire task. In such settings, learning individual primitives for each stage that succeed with a high enough rate to perform a complete temporally extended task is impractical: if each stage must be completed successfully and has a nonnegligible probability of failure, the likelihood of successful completion of the entire task becomes negligible. Therefore, successful controllers for such multistage tasks must be able to recover from failure and compensate for imperfections in low-level controllers by smartly choosing which controllers to trigger at any given time, retrying, or taking corrective action as needed. To this end, we describe an imitation learning system that uses vision-based policies trained from demonstrations at both the lower (motor control) and the upper (sequencing) level, present a system for instantiating this method to learn the cable routing task, and perform evaluations showing great performance in generalizing to very challenging clip placement variations.

I. Introduction

Complex, multistage robotic manipulation tasks often arise in robotic manipulation applications of practical interest: from home robots that need to prepare a meal to industrial robots that need to assemble a complex device. Many of the tasks we might want to automate in these settings consist of complex low-level behaviors and also require these behaviors to be sequenced appropriately to perform the overall task. This presents a major challenge: when a robot naïvely executes a sequence of primitive behaviors to perform a complex task, the probability of failing at the task grows multiplicatively with each stage. Advances in perception, control, and robotic learning can enable each stage of a task to be more performant, but as long as sequencing stages naïvely leads to such difficulties, elaborate multistage tasks that consist of a sequence of individually difficult primitives will remain out of reach. In this article, we examine how hierarchical imitation learning (IL) with levels of learned primitives and high-level sequencing can address this issue, in the context of a difficult multistage cable routing task. Our focus is on providing for robustness at both levels of the hierarchy, not by trying to construct perfect robotic skills that never fail, but by endowing both levels of the hierarchy with the ability to correct and recover from mistakes. We study the problem of multistage cable routing, where a robot routes a cable through a series of clips (see Fig. 1). This task is representative of commonly occurring scenarios in manufacturing and maintenance, where a robot might need to route cables or hoses, and provides an interesting challenge for robotic manipulation: each stage of the cable routing task requires handling the deformable cable, reasoning about complex contact patterns between the cable and the clip, accounting for deformations, and also observing that the cable has been clipped into each clip successfully. At the same time, the higher level sequencing of primitives might require retrying a clipping motion, securing the cable by pulling it taut, and dynamically deciding when to advance to the next clip. This task is, therefore, both practically relevant for industrial applications, and captures many of the essential characteristics of multistage manipulation discussed in the preceding paragraph, with challenges and ambiguity at both the lower and upper level necessitating intelligent controllers that combine perception and control, and recovery from failure.