Detection and Segmentation of Unlearned Objects in Unknown Environment

Detecting and segmenting unlearned objects in unknown environment is a very important visual perception ability to enhance industrial intelligence. In this article, we present a novel conditional random field model integrating unimodal and cross-modal terms for detecting and segmenting object instances without knowing their categories and without sampling extra proposals. This model takes a paired image and point cloud as input, from which we first develop a set of novel category-independent features to distinguish objects. Then, a set of unary, pairwise, and higher order potentials are designed according to these category-independent features, and the cross-modal potential is introduced as a novel global constraints to keep the spatial consistency in both 2-D and 3-D modalities. In this novel model, the unlearned object detection and segmentation is treated as the process of pixel labeling. Thus, adjacent or occlusion object instances can also be separated efficiently from a labeled map. By comparison with the baseline methods, experimental results on a public RGB+D dataset show that the proposed model can obtain better performance with improved precision and recall rate. Moreover, we use the proposed method in a real industrial scene and achieve satisfactory performance.