Gesture based control of IRB1520ID using Microsoft's Kinect

This research develops a Gesture based technique to manipulate a ABB's industrial robot IRB1520ID using Microsoft's Kinect Sensor. Kinect is used for acquiring skeletal data and coordinates of each joint of a person and send live frames to MATLAB for processing. Our MATLAB code will calculate joint angles and send signals to the robot. The robot's controller will accordingly trigger routines to move the end-effector in the desired direction, for example if the hand is moving towards right, the robot will also move towards right and vice versa. Communication between MATLAB and the robot's controller is accomplished by using an Arduino Nano which generates the required I/O signals understood by the robot. This technique is very effective since it takes real time motion of the hand and connects it to motion of the robot.