Aligning Objects as Preprocessing Combined with Imitation Learning for Improved Generalization

Imitation learning method transfers human behavior to the robots or machines. This method aims to allow robots or machines to learn by observing tasks performed by human operators and imitating these tasks, rather than direct programming. ACT as an imitation learning method shows the high capability for automating dexterous manipulation tasks. From the viewpoint of industrial application, pose of the target object will be varied. However, even if only for the initial object pose variation, imitation learning method like ACT usually needs a lot of demonstration data that covers pose variation to train the policy that can generalize for. Collecting large demonstration dataset takes many efforts. This study created an object pick-and-place controller to eliminate pose variation as a preprocess step with YOLOv8, which is a recent object detection technique. The preprocess step automatically moves the object to a specific position and eliminates the pose variation. We show that our system effectiveness on the randomly placed bag opening task that requires both generalization for object pose variation and dexterous bimanual manipulation. The bag opening task was conducted with ACT and preprocess applied ACT methods, and the results were evaluated to examine the effect of the preprocess method to generalization process.