Stereo Vison and Mask-RCNN Segmentation Based 3D Points Cloud Matching for Fish Dimension Measurement

Fish dimensions information is of great significance for aquaculture. Traditional manual measurement is labour intensive, time-consuming and may infulence the growth of fishes. A Stereo vison and Mask-RCNN based Dimension Measurement Scheme(SMDMS) is proposed to get dimensiones of mutiple fishes without constraining their movement. The SMDMS is consist of three parts including stereo vision, fish instance segmentation, 3D points cloud process. In stereo vision, a binocular camera is calibrated and the captured images are rectified to row-aligned. The 3D information is subsequently calculated by 3D reconstruction with a stereo match method. In detection and segmentation part, a Mask Region Convolution Neural Network(Mask-RCNN) is trained to achieve fish detection and initial segmentaion simultaneously. The segmentation is then refined with morphological process and an interactive segmentation method called GrabCut. In the last part, the 3D points cloud is extracted from the 3D information according to the refined segmentation and some coordinate transformations are subsequently applied to make the estimation more robust for fishes with various orientations and positions. The transformation is calculated by some fitting process. The estimation result is compared with the manual measurement of fishes with different dimensions. It turned out that the proposed scheme achieved high accuracy both in length and width estimation.