A Robotic Manipulation Framework Based on Large Model and Behavior Cloning

To enhance the generalizability of intelligent robots to perform tasks, a robotic manipulation framework based on large model and behavior cloning is proposed. First, the vision-language-grasping large model-based visual processor is used to locate and generate grasping pose for the target object. After the robotic arm grasps the object according to this pose, it moves to the initial starting position of the Relative Pose-Action Chunking with Transformers (RP-ACT) network, which is determined based on the data collected during the human demonstration phase. Then, at each time step, the current pose of the robotic arm's end-effector relative to the target object is calculated and input into the trained RP-ACT network, and the robotic arm's pose is updated based on the network's output. For five typical tasks, real-robot experiments were conducted. Results show that, under varying target positions, the average success rate across these five tasks was 83.6 %. When background interference objects were present, the average success rate across the five tasks reached 79.2%. The proposed framework was proven effective in enhancing scene generalization with low data volume and short training time.