An Autonomous Eye-in-Hand Robotic System for Elevator Button Operation Based on Deep Recognition Network

Autonomous elevator button operation is an indispensable function for robot-elevator interaction, which has long been considered an intelligent solution for multifloor navigation of mobile robots. In this article, we present an autonomous robotic system with an eye-in-hand configuration to address the button operation problem. First, we develop a deep neural network for simultaneous button detection and character recognition, which provides accurate and robust perception inputs for the button operation system. Second, we present a button pose estimation algorithm with consideration of perception uncertainties, and the button pose is accurately estimated by fitting a least-uncertainty model. Based on the perception and pose estimation algorithms, a coarse-to-fine control schema is then proposed for driving the manipulator to accomplish the button operation task. Experimental results show that the proposed perception algorithm can outperform state-of-the-art methods in both recognition accuracy and running efficiency. The pose estimation and control schema also demonstrate their effectiveness in real-world button operation tasks. The data and code are available at our project webpage.