Rapid Constrained Object Motion Estimation based on Centroid Localization of Semantically Labeled Objects

Autonomous and semi-autonomous robot manipulation systems require fast classification and localization of objects in the world to realize online generation of motion plans and manipulation waypoints in real-time. Furthermore, constraints and estimated plausible motions of objects of interest in space is paramount for autonomous manipulation tasks. For nongrasping tasks like pushing a box or opening an unlatched door, physical properties such as the center of mass and location of constraints like hinges or bearings must be considered. This paper presents a methodology for rapidly inferring constraints and motion plans for objects of interest to be manipulated. This approach is based on a combination of object detection, instance segmentation, localization methods, and algebraically relating different semantically labeled objects. These methods for motion estimation are implemented on a color-depth camera (RGB-D) and a 7 degree-of-freedom serial robot arm. The algorithm’s performance is evaluated through different arm poses, assessing both centroid accuracy and estimation speed, and motion estimation performance. Algorithms are tested on an exemplar problem consisting of a block constrained on a dual linear rail system, i.e., constrained linear motion. Experimental results showcase the scalability of this approach to multiple classes with sublinear slowdowns and linear motion plan direction errors as low as 1.23E–4 [rad]. The manuscript also outlines how these methods for rapid constrained object motion estimation can be leveraged for other applications.