SAMS : One-Shot Learning for the Segment Anything Model Using Similar Images

The field of computer vision is currently transitioning from closed-set to open-set tasks. Vision foundation models have already demonstrated success in open-set scenarios. Building on these models, the utilization of a feature supervision framework can further enhance results. Our paper introduces a new method called SAMS (Segment Anything Model using Similar Images), which is a type of feature supervision framework. It is designed to segment specific masks from visual supervision features. Our framework comprises a pre-trained segmentation model and an efficient, novel prompt generation model capable of generating new prompts based on pre-extracted image features. This innovation eliminates the need for manually crafted prompts in the mask generation phase by integrating the principles of one-shot or few-shot learning with visual instructions from similar images. The effectiveness of the SAMS method is evident in its performance across various tasks, particularly in open-set tasks where traditional models tend to struggle. The pretrained model not only achieves impressive mean Intersection over Union (mIOU) scores without incurring additional time loss, but also demonstrates potential for further improvement through targeted module training.