SegBuilder: A Semi-Automatic Annotation Tool for Segmentation

This paper addresses the problem of image annotation for segmentation tasks. Semantic segmentation involves la-beling each pixel in an image with predefined categories, such as sky, cars, roads, and humans. Deep learning mod-els require numerous annotated images for effective training, but manual annotation is slow and time-consuming. To mitigate this challenge, we leverage the Segment Anything Model (SAM) [23] - a vision foundation model. We intro-duce SegBuilder, a framework that incorporates SAM to au-tomatically generate segments, which are then tagged by human annotators using a quick selection list. To demon-strate SegBuilder's effectiveness, we introduced a novel dataset for image segmentation in underwater environments featuring animals such as sea lions, beavers, and jellyfish. Experiments on this dataset showed that SegBuilder signif-icantly speeds up the annotation process compared to the publicly available tool, Label Studio [31]. SegBuilder also includes a free-form drawing tool, allowing users to cre-ate correct segments missed by SAM. This feature is par-ticularly useful for scenes with shadows, camouflaged ob-jects, and part-based segmentation tasks where SAM falls short. Experimentally, we demonstrated SegBuilder's efficacy in these scenarios, showcasing its potential for generating pixel-wise annotations crucial for training robust deep learning models for semantic segmentation.