Automatic in-situ instance and semantic segmentation of planktonic organisms using Mask R-CNN

Planktonic organisms form the principal source for consumers on higher trophic levels in the food chain. Studying their community dispersion is vital to our understanding of the planet's ecological systems. With the recent technological advancements in imaging systems, capturing images of planktons in-situ is made possible by embedding mobile underwater robots with sophisticated camera systems and computing power that implement deep machine learning approaches. Efforts of applying deep learning methods to plankton imaging systems have been limited to classification, while detection and segmentation has been left to traditional methods in this context. There is a variety of publicly available datasets made suited for planktonic species classification. These datasets consist of images of individual species. Thus, they do not represent the actual environment, which is usually given by a scene representation more suited for object localization, detection and semantic segmentation. In this paper we propose a novel custom dataset [1] from planktonic images captured in-situ in a lab environment suited for supervised learning of object detection and instance segmentation. The data is tested in experiments using the state-of-the-art deep learning visual recognition method of Mask R-CNN. The experiment results show the potential of this method and create a baseline analysis module for real-time in-situ image processing. We provide a comparison of how the method is performing when trained on automatically processed and annotated images from existing segmentation frameworks using traditional methods. This comparison illustrates the importance of utilizing proper data and the potential for success if provided¹¹All results, code and metrics used for the experiments are provided in: https://github.com/AILARON/Segmentation