Progressive Transfer Learning for Dexterous In-Hand Manipulation with Multi-Fingered Anthropomorphic Hand

Dexterous in-hand manipulation poses significant challenges for a multi-fingered anthropomorphic hand due to the high-dimensional state and action spaces, as well as the intricate contact patterns between the fingers and objects. Although deep reinforcement learning has made moderate progress and demonstrated its strong potential for manipulation, it faces certain challenges, including large-scale data collection and high sample complexity. Particularly in scenes with slight changes, it necessitates the re-collection of vast amounts of data and numerous iterations of fine-tuning. Remarkably, humans can quickly transfer their learned manipulation skills to different scenarios with minimal supervision. Inspired by the flexible transfer learning capability of humans, we propose a novel framework called Progressive Transfer Learning (PTL) for dexterous in-hand manipulation. This framework efficiently utilizes the collected trajectories and the dynamics model trained on a source dataset. It adopts progressive neural networks for dynamics model transfer learning on samples selected using a new method based on dynamics properties, rewards, and trajectory scores. Experimental results on contact-rich anthropomorphic hand manipulation tasks demonstrate that our method can efficiently and effectively learn in-hand manipulation skills with just a few online attempts and adjustment learning in the new scene. Moreover, compared to learning from scratch, our method significantly reduces training time costs by 85%.