Enhanced No-Code Finger-Gesture-Based Robot Programming: Simultaneous Path and Contour Awareness for Orientation Estimation

The programming of industrial robots necessitates specialized expertise and significant time and effort, particularly for small batch sizes. However, with the increasing demand for agility in production, the solutions used for robot programming have evolved significantly. Intuitive robot programming systems based on diverse concepts have been introduced to facilitate rapid deployment of robot systems. One such approved concept is no-code robot programming with finger-based gesture. In this concept, non-expert users draw a robot path via finger movement, which is subsequently translated into robot programming language to facilitate the corresponding movement. A significant challenge associated with this method is the valid replication of the corresponding robot Tool Center Point (TCP) orientation. Reachability issues, non-compliant hand contortions, and sensor occlusions make it difficult to directly derive the robot’s TCP orientation from the finger’s orientation. This work presents two novel approaches for estimating robot orientation using numerical analysis and point cloud information for finger-based robot programming without requiring prior knowledge. The first approach utilizes numerical analysis to estimate the relative robot orientation based on the geometry of the trajectory. In contrast, the second approach uses point-cloud to derive the robot orientation based on the object contour. Input shaping algorithms are employed and evaluated to reduce the divergence in the orientation estimations. Experiments demonstrate the effectiveness of the proposed approach utilizing a low-cost camera as a cost-efficient alternative to existing no-code programming strategies, potentially accelerating the real-world deployment of robotic applications in industrial environments.