Few-Shot Parameter Efficient Finetuning for SAM in Salient Steel Surface Defect Detection

Current deep learning methods for strip steel defect detection prioritize accuracy and computational efficiency but struggle to generalize to new production environments with unseen defect types. While visual foundation models like segment anything model (SAM) offer broader generalization through pretrained visual knowledge, they face significant domain shifts when applied to industrial defect imagery. To address this, we propose AdaptedSAM, a parameter-efficient few-shot fine-tuning framework for SAM that optimizes adapter placement and feature interaction. We first demonstrate that adapter positioning within the encoder critically impacts few-shot adaptation performance. Building on this, we introduce an efficient global optimizer that dynamically allocates adapter weights via cross-block communication across transformer layers. Furthermore, we develop an adapted mask decoder to enhance detailed mask information. Our adaptedSAM achieves state-of-the-art defect detection with precise boundaries in five-shot learning, outperforming 12 state of the art (SOTA) models on the SD-saliency-900 steel dataset while ensuring computational efficiency. Cross-domain evaluations on six benchmarks (industrial defects, remote sensing, medical imaging, natural scenes) demonstrate robust generalization.