Reliable hand camera based book detection and manipulation in library scenario

In this paper a new approach for detection, grasping and placing of books is described. The location as well as book's size, color and textures are unknown. Therefore the autonomous book detection and grasping is a big challenge. In this work, the proposed approach is implemented in the scope of a library scenario in which a service robotic system operates. The proposed approach uses stereo vision for coarse approach to the next book, which should be grasped, and a hand camera (mounted on the gripper) based positioning strategy to achieve a high success rate of grasping. The full concept was implemented and tested on the service robot FRIEND. It should support a handicapped person to work and to perform all manipulative tasks. These contain the autonomous book detection on a book cart, the grasping and placing of the book on a book holder and later onto the cart again. The benefit of our approach is documented based on experimental results.