Camera-LiDAR Data Fusion for Autonomous Mooring Operation

The use of camera and LiDAR sensors to sense the environment has gained increasing popularity in robotics. Individual sensors, such as cameras and LiDARs, fail to meet the growing challenges in complex autonomous systems. One such scenario is autonomous mooring, where the ship has to be tied to a fixed rigid structure (bollard) to keep it stationary safely. The detection and pose estimation of the bollard based on data fusion from the camera and LiDAR are presented here. Firstly, a single shot extrinsic calibration of LiDAR with the camera is presented. Secondly, the camera-LiDAR data fusion method using camera intrinsic parameters and camera to LiDAR extrinsic parameters is proposed. Finally, the use of an image-based segmentation method to segment the corresponding point cloud from the fused camera-LiDAR data is developed and tailored for its application in autonomous mooring operation.