Annotation-Free Object Segmentation Using Zero-Shot Detection and SAM for Fruit Classification

This paper presents a comparative analysis of ten deep learning architectures for automated fruit quality classification. The dataset includes six fruit categories—Apple, Banana, Guava, Lime, Orange, and Pomegranate—each labeled as Good or Rotten. Two experimental setups were evaluated: (1) full-frame image classification using frozen pretrained models, and (2) a pipeline combining zero-shot object detection (Florence2, Grounding DINO, YOLO-World) with segmentation (SAM) followed by classification. Results showed that Florence2+SAM and Grounding DINO+SAM significantly improved classification accuracy, achieving up to 98.9% with ResNet152, outperforming the best full-frame models. YOLO-World underperformed due to imprecise localization that degraded segmentation quality. ResNet152 consistently achieved the highest accuracy across all pipelines, demonstrating superior representational capacity and stable fine-tuning behavior. These findings highlight the effectiveness of segmentation-aided classification and the impact of model architecture in quality assessment tasks. Limitations related to dataset diversity, computational cost, and segmentation reliability are also discussed.