Collaborative Operation of Robotic Manipulators with Human Intent Prediction and Shared Control

This paper develops technical tools to facilitate the collaborative operation of robot manipulators by providing human operators an effective way to remotely command a manipulator for object inspection and handling applications by using computer vision, human intent prediction, and human-machine shared control. First, an operational interface and robot motion control implementation based on a low-cost, general-purpose joystick game controller is presented. Then, a design of a robot end-effector that can help realize motion based on computer vision feedback is proposed and developed. Utilizing the capabilities of computer vision, an effective control law to keep a detected object at the center of the camera view by adjusting end-effector orientation is developed. Lastly, human intent prediction and human-machine shared control for this inspection and handling robotic operations are described. In the shared control system, the human operator takes initiatives; however, the system can facilitate task execution by having the end-effector orientation controlled automatically based on the human's intent. To corroborate the proposed methods and design, a custom end-effector which can be mounted on an industrial robot is developed and the robot system is integrated to be commanded by a general-purpose game controller. Experimental results on an object inspection and handling experiment are provided to show the effectiveness of our approach.