Instance Segmentation of Nematode Image Based on Transformer with Skeleton Points

Study of Nematode phenotyping is prevalent in biological field due to its wide applicability. We can obtain essential information through monitoring morphological changes of nematodes in genetic studies, drug testing and so on. Currently, image intelligent analysis has become a key method to capture the information of nematode morphology, in which image segmentation technology plays an important role. In the task of nematode image segmentation, the occlusion among nematodes often occurs, which brings huge challenges to accurate nematode instance segmentation. To mitigate the occlusion problem, we propose a Transformer-based nematode segmentation network that combines both mask prediction and skeleton points prediction. Specially, in the network's head structure, we simultaneously consider mask prediction (global features) and sequential skeleton points prediction (local significant features), which enriches the contextual information and could separate the overlapping nematodes more exactly. We have validated the effectiveness of our method on two public nematode datasets: the Broad Bioimage Benchmark Collection 010 (BBBC010) and the Haemonchus contortus dataset. Our method outperforms mainstream segmentation approaches including Mask R-CNN, Cascade Mask R-CNN, MS R-CNN, PointRend, HTC, and SOLOv2 in terms of segmentation accuracy (Average Precision).