MLGTT: An Open-Source Tool to Generate Camera-Relative Ground Truth for Monocular Localization

Ground truth is essential in Computer Vision problems as it establishes a baseline for error thresholds in practical applications. This paper presents the Monocular Localization Ground Truth Tool (MLGTT), a modular ground truth seeking tool for camera pose calculations, aimed at obtaining accurate metrics to verify the performance of localization algorithms while isolating intrinsic image error factors. Our tool leverages a Perspective-n-Point (PnP) algorithm, and a Random Sample Consensus (RANSAC) iterative algorithm, which approximates the best fit model, represented by the highest number of inliers and the lowest reprojection error. To utilize this PnP algorithm, we require a predetermined set of 3D points selected on an object's CAD model, 2D points from an image of the object that are manually selected in the tool, and a camera model that includes distortion parameters for handling unrectified images. These parameters yield a transformation matrix from the camera to the object. The MLGTT reprojects these points onto the canvas using the best transformation matrix from PnP. When paired with a rasterizer tool, the MLGTT can synthesize an image using the calculated transformation matrix that overlays on top of the original image. A slider tool is also implemented into the MLGTT which provides the user with a better visual aid for verifying localization fidelity, by blending the original image with the rendered image with a slider. This visual verification helps ensure the accuracy of the pose estimation to the level of accuracy of the human eye. The MLGTT allows human perception to aid in the findings of the ground truth, which removes error found in other methods. We introduce our characterization of user and pixel error, quantified through experiments with individuals both familiar and unfamiliar with the MLGTT or computer vision generally. We will also discuss reprojection error on different screen resolutions and how they might affect the findings. Additionally, we quantify the ground truth error in free spaces as a function of the resolution and field of view (FOV) of the camera that captured the image. Findings include demonstrating the tool's precision and ease of use compared to traditional external ground truthing methods, such as Vicon or AprilTags. These traditional systems typically require external markers and complex setups, whereas the MLGTT operates without such additional elements This makes the MLGTT more versatile and easier to deploy in various environments, demonstrated by testing on Mars 2020 images. Originally designed to provide independent ground truth to verify our localization algorithms for the Mars Sample Return (MSR) campaign, the MLGTT has been adapted for multiple different applications and can be applied generally to any monocular imagery of known hardware with a calibrated camera. This software will be released as an open-source project along with this paper.