Robotic Manipulations of Cylinders and Ellipsoids by Ellipse Detection With Domain Randomization

A lot of objects commonly found in industrial and household environments are represented by cylindrical shapes (flange plates) and ellipsoids (oranges). The visual tops of cylindrical objects and the outlines of ellipsoids are approximately formed by elliptic shape primitives. Thus, a robot can manipulate these objects by ellipse detection. However, it poses a challenge for existing methods to detect ellipses that are largely interfered in cluttered environments. The supervised learning method provides a potential solution for ellipse detection in such environments, but manually labeling the ground truth for training the learning model costs much time, money and workload. In this work, we propose a novel approach to employing three-dimensional point-cloud models of cylindrical objects to construct synthetic images with ellipses based on domain randomization technology for training the supervised learning model. Using synthetic data generated in this manner, we build an end-to-end deep neural network with a detection backbone, rotating filters and the rotated region proposal net to do ellipse detection. To our knowledge, this is the first supervised learning model trained only on synthetic data for ellipse detection, enabling a robot to grasp cylindrical and ellipsoid objects. To demonstrate the capabilities of the proposed detector, the performance of the proposed method is superior to those of two state-of-the-art detectors on synthetic and public datasets. The proposed model for ellipse detection and data generation pipeline based on domain randomization in a simulation are evaluated by a series of robotic manipulations implemented in real application scenarios. The results illustrate a high success rate on real-world grasp attempts despite having only been trained on a synthetic dataset.