Vision-Based Localization and Tracking of Objects Through Robotic Manipulation

Among all the technological developments in the past decade, innovations in robotics are one of the most significant. Robots can now carry out both simple and complex jobs from laboratory to industrial settings with accuracy and efficiency. With the help of vision, robotics and AI offer humans numerous opportunities. This research aims to illustrate the development of a vision-based robot manipulation system that can locate and track a target object in real time. The system employs a depth camera, a UFactory xArm robot, a pre-trained model, and OpenCV to receive vision sensory input, recognize objects, and generate interactive coordinates for each target object. Using a 5-degree-of-freedom robot (xArm-5) and a RealSense depth camera, a thorough experiment was conducted to validate the proposed system’s performance. Using the detection accuracy of 75.2% and average depth accuracy of 94.5%, the proposed system performs stably and can successfully track target objects via robot manipulation with 30 frames per second. This technology has tremendous promise in the fields of exploration, mobile robots, and assistive robotic systems.