Visual Sensor Target Tracking and Localization Method for Automatic Excavators

The efficacy of visual measurement-based control systems for excavator trajectory operations hinges on the reliability and accuracy of visual sensors, dictated by the identification of target feature points. This encompasses two pivotal aspects: the continuous and accurate tracking of markers, essential for the visual sensor system’s reliability, and the precise localization of marker feature points, critical for determining the excavator’s operational pose. Addressing the shortcomings of current methodologies, we introduce an adaptive Region of Interest (ROI) method, employing a predictor that combines marker motion and size change information to forecast the marker’s future location and size, thereby minimizing background noise and ensuring continuous marker tracking. A corrector subsequently refines the ROI, leveraging marker characteristics to preserve marker integrity and optimize ROI determination. Our experiments in various complex operational environments demonstrate the superiority of the adaptive ROI method over conventional and deep learning approaches in enabling the maintenance of continuous and precise marker tracking. This ensures the reliability of visual measurements without reliance on extensive datasets and high computational power. Furthermore, to mitigate feature point localization noise, we propose a variable-quality color target localization method that allocates varying qualities to pixels based on their Hue component, markedly diminishing noise in comparison to traditional constant-quality methods. This thorough study establishes a foundation for the visual measurement of automatic excavator pose and the closed-loop control of operation trajectories through visual measurement. Additionally, it serves as a reference for target tracking and localization in similar application scenarios.