Robotic Manipulation of Hand Tools: The Case of Screwdriving

Despite decades of steady research progress, the robotic hand is still far behind the human hand in terms of dexterity and versatility. A milestone in this quest for human-level performance will be possessing the skills of manipulating hand tools, for their non-trivial geometries and for the intricacies of controlling their contact-based interactions with objects, which are the final targets of manipulation. This paper investigates screwdriving by a robotic arm/hand pair, dealing with the chain of contacts connecting the substrate, screw, screwdriver, and fingertips. Considering rolling contacts and finger gaits, our force control scheme is derived through backward chaining to leverage the dynamics of the screwdriver and arm/hand. To maintain the fastening effort, estimations are carried out sequentially for the screwdriver’s pose via optimization under visual and kinematic constraints, and for its applied wrench on the screw via solution drawing upon dynamics. This wrench, adjusted based on position/force feedback, is mapped by the grasp matrix to the desired fingertip forces, which are then used for computing torques to be exerted by the arm and hand to close the loop. Simulation and experiments with a Shadow Hand have been conducted for validations.