Auto-Prompting SAM for Container Detection and Localization in Container Yards

Container detection and localization are crucial for the automated loading and unloading of containers. In automating rubber-tired gantry cranes (RTGs) for container handling, small- and medium-sized terminals are increasingly favoring for vision-based solutions over expensive LiDAR (Light Detection and Ranging) systems. However, conventional image processing-based methods fail to meet robustness and real-time requirements in practical settings. Meanwhile, deep learning-based object detection approaches offer greater adaptability to complex environments but are limited by their dependence on large-scale datasets and insufficient localization accuracy. To address these challenges, this paper presents a three-stage container detection and localization method for RTGs, exploiting a Segment Anything Model and image processing. First, an Auto-Prompting Segment Anything Model (AP-SAM) is proposed for the first time. It strengthens the similarity between reference and target features by training a feature adaptor module that uses only a single reference image with segmentation labels. Next, local peak points of the similarity map are selected as prompts, guiding the SAM to achieve initial container segmentation. Subsequently, we design an image processing method considering a container’s structural and geometric characteristics to detect its top surface, castings, and lock holes. Furthermore, we propose a novel estimation strategy to infer the positions of unknown lock holes on the detected ones, significantly improving the location accuracy. Extensive experiments show that our proposed method achieves a container detection precision of 93.3% and an average location error of just 5.55 pixels, surpassing existing methods.