Binocular Vision-based Speed and Separation Monitoring of Perceive Scene Semantic Information

Among the existing methods for realizing the safety of human-robot collaboration (HRC) under speed and separation monitoring (SSM), there are large safety zones, poor real-time performance, lack of information perception of the collaboration environment, and overly complex methods. Considering the characteristics of the binocular camera such as simple structure, low hardware cost, and strong ability to acquire depth information, in this paper, we propose binocular vision-based SSM(BV-SSM) to realize the tracking of robot joint coordinates by vision algorithms. The proposed BV-SSM adopts object detection algorithms trained on our self-constructed dataset to enhance the robot’s ability to perceive scene information, which not only reduces the computational complexity but also obtains an accurate and reasonable minimum safe distance by analyzing the speed directions of possible collisions under different motion states. Finally, the experimental tests are conducted on a six-degree-of-freedom industrial robotic arm with a ZED2i binocular camera, and the experimental results show that the proposed BV-SSM method can obtain an accurate and rationalized safety zone in real-time in HRC.