A Novel Deep Reinforcement Learning-Based Path/Force Cooperative Regulation Framework for Dual-Arm Object Transportation

Dual-arm robots, with their high flexibility and broad operational range, are widely used in industrial and household transportation tasks. However, their high degrees of motion freedom and the closed-chain constraints formed during transportation pose challenges for path planning and force control. This study proposes a dual-agent control framework based on the Soft-Actor-Critic (SAC) algorithm of Deep Reinforcement Learning (DRL), where one agent is responsible for path planning and the other handles force control. This framework enables dual-arm robots to achieve dynamic obstacle avoidance, avoid unsolvable configurations and singularities, and generate efficient and smooth paths, while also fulfilling internal force tracking requirements based on task demands. Additionally, it addresses the gap in transferring the force control agent from simulation to real-world applications through a state mapping network, and the force control agent does not require retraining for different objects. Finally, the effectiveness of the proposed framework is validated through two scenes, including multiple bookshelves stacking and dynamic obstacle avoidance during the box transportation. Validation was also carried out with objects of different geometries and weights. Note to Practitioners—The motivation of this study is the path planning and force tracking control problem for dual-arm transportation. Existing methods rarely provide a comprehensive framework to address the task of dual-arm transporting different objects in dynamic environments. A DRL-based dual-agent control framework is proposed in this study, which simultaneously achieves task-compliant path planning and force tracking control. The trained agents exhibit strong generalization capabilities and stability, allowing them to handle objects of different geometries, weights, and stiffness without retraining. The limitation lies in the lack of methods for generating dual-arm initial grasping actions for complex-shaped objects. Future research will integrate image recognition techniques to enable grasping action generation for objects in diverse poses, further enhancing the applicability of the proposed framework.