Crop Classification in UAV Remote Sensing Images Using Segment Anything Model

With the rapid development of Unmanned Aerial Vehicle (UAV) remote sensing technology, the acquisition of high-resolution and large-scale crop data has become increasingly convenient. Compared to satellite remote sensing, UAV imagery plays a significant role in crop classification and yield prediction. Traditional supervised learning methods rely heavily on large volumes of annotated data, which, while effective in improving model performance, are often limited by the high cost of annotation—posing a challenge for wide-scale application in the agricultural domain. Therefore, exploring efficient and low-cost pre-trained model approaches is of great importance for advancing UAV remote sensing applications in crop classification.This study utilizes a high-resolution UAV remote sensing dataset covering agricultural areas in Xingren City, Guizhou Province, China. The dataset comprises two ultra-large remote sensing images, with resolutions of 47,161×50,141 and 77,470×46,050 pixels, respectively. Due to the massive size of these images, they cannot be directly fed into neural networks. Thus, a sliding window cropping strategy was employed, segmenting the images into multiple sub-images of 512×512 pixels. After cropping, the dataset contains a total of 6,872 sub-images, with the training and testing sets split approximately at a ratio of 4:1. This dataset provides ample remote sensing imagery for model training and evaluation, ensuring reliability and generalization in the classification tasks.This study fine-tunes the Segment Anything Model (SAM) to analyze UAV-based multi-band remote sensing time-series imagery for the purpose of crop classification. Originally pretrained for generic image segmentation tasks, SAM is specifically fine-tuned here to identify crop types in high-resolution UAV imagery. During the fine-tuning process, the model leverages both temporal and spectral features of the remote sensing data to enhance its classification accuracy for Coix seed, corn, and flue-cured tobacco.The fine-tuned SAM model demonstrates broad applicability in agricultural domains, including crop classification and yield prediction, and maintains a recall rate of no less than 89% even under limited annotation scenarios. Experimental results indicate that SAM exhibits strong transferability in UAV-based crop image segmentation tasks, confirming its great potential in agricultural geographic information applications.