Research and application of indoor 3D object detection based on Lidar and Monocular camera

With the increasing degree of automation of indoor mobile robots, the perception requirements of indoor objects are also higher and higher, which needs to clarify the types of objects and master their three-dimensional size information. In this scheme, the combination of lidar and monocular cameras is adopted to compensate for the deficiency of a single sensor. The data-cascading network structure is adopted. Firstly, the YOLOP model detects and segments the image in two dimensions. After that, the PointNet model performs instance segmentation on the object's point cloud, followed by 3D bounding box regression to identify the target object. The network is used to determine the types of objects and obtain their three-dimensional size information. Through data set and field verification, the proposed data cascade network is superior to most previous methods. It can accomplish the detection task of indoor 3D objects based on lidar and monocular cameras.