Real-Time Servo Control with Gesture Recognition on Nvidia Jetson Using CNNs

Artificial intelligence (AI) algorithms are increasingly utilized across various domains, including real-time industrial control. This paper presents a real-time servo system control using hand gestures, detected by a convolutional neural network (CNN) implemented on an NVIDIA Jetson Edge device. The recognized gestures are transmitted via a deterministic industrial protocol (Profinet) to a programmable logic controller (PLC), which drives the servo system through motion function blocks (MFBs). This approach demonstrates significant improvements in real-time control for human-machine interaction (HMI) applications compared to direct CNN implementations on PLCs.