A Model Predictive Control Approach for Vision-Based Object Grasping via Mobile Manipulator

This paper presents the design of a vision-based object grasping and motion control architecture for a mobile manipulator system. The optimal grasping areas of the object are estimated using the partial point cloud acquired from an onboard RGB-D sensor system. The reach-to-grasp motion of the mobile manipulator is handled via a Nonlinear Model Predictive Control scheme. The controller is formulated accordingly in order to allow the system to operate in a constrained workspace with static obstacles. The goal of the proposed scheme is to guide the robot's end-effector towards the optimal grasping regions with guaranteed input and state constraints such as occlusion and obstacle avoidance, workspace boundaries and field of view constraints. The performance of the proposed strategy is experimentally verified using an 8 Degrees of Freedom KUKA Youbot in different reach-to-grasp scenarios.