Large-Deformation Non-Rigid Body Shaft-Hole Assembly Method Based on Visual and Force Information Fusion

This paper aims at the challenges of complex material deformation and non-linear contact mechanics in non-rigid body shaft-hole assembly tasks, and proposes an innovative method that integrates multi-modal perception and hierarchical reinforcement learning. By decomposing the assembly process into two stages: pose adjustment and hierarchical reinforcement learning insertion, the task complexity is effectively reduced. At the perception level, the YOLO-World open-vocabulary detection model is used to locate the center of the hole, and the SAM-Track segmentation and tracking model is combined to obtain the deformation information of the non-rigid body shaft in real time. A multi-modal perception system without the need for annotated data is constructed, enabling pixel-level deformation parameter extraction and 3D pose calculation. In terms of control strategy, the insertion process is innovatively divided into two stages: shallow insertion and deep insertion. Independent SAC reinforcement learning networks are designed to optimize the impedance control parameters respectively, and hierarchical compliant control is achieved through a hierarchical reward function. Experimental results show that under the conditions of 0.20mm and 0.40mm shaft-hole clearances, the assembly success rate of this method reaches 100%, the average assembly time is 27.9s and 26.3s respectively, and the contact force and torque are stabilized within 2.5N and $1.8N \cdot m$ respectively. Through the collaborative optimization of visual-force fusion perception and hierarchical reinforcement learning, it provides a new technical path for non-rigid body shaft-hole assembly in flexible manufacturing scenarios.