Vision-Based Pick and Place Control System for Industrial Robots Using an Eye-in-Hand Camera

In this paper, we present a vision-based pick-and-place control system for industrial robots using an eye-in-hand camera. In industry, using robots with cameras greatly improves efficiency and performance. Previous studies have focused on utilizing robotic arms for the pick-and-place process in simulated environments. The challenge when experimenting with real systems lies in aligning the coordinate systems between the robot and the camera, as well as ensuring high data accuracy during experimentation. To address this issue, our research focuses on utilizing a low-cost 2D camera combined with deep learning algorithms mounted on the end-effector of the robotic arm. This study is evaluated in both simulation and real-world experiments. We propose a novel approach that combines the YOLOv7 (You Only Look Once V7) deep learning network with GAN (Generative Adversarial Networks) to achieve fast and accurate object recognition. This system uses deep learning to process camera data to extract object positions for the robot in real-time. Due to its advantages of fast inference and high accuracy, YOLO is applied as the baseline for research. By training the deep learning model on diverse objects, it effectively recognizes and detects any object in the robot’s workspace. Through experimental results, we demonstrate the feasibility and effectiveness of our vision-based pick-and-place system. Our research contributes an important advancement in the field of industrial robots by showcasing the potential of using a 2D camera and an integrated deep learning system for object manipulation.