Kinect sensor based real-time robot path planning using hand gesture and clap sound

Many indoor applications such as cleaning, object recognition etc. are replaced by a robot. For indoor applications if we could control the path of the robot using gesture and clap sounds. In this composition, this work proposes and implements an advanced method to control mobile robot in real-time using short adjustment of clap sound and hand gesture commands from Microsoft Kinect sensor linked to a laptop/PC and mobile robot is connected via RF link. The commercial hardware is predicated on Microcontroller code. To avoid unnecessary motion of the robot by distinguishing the motion by distinctive persons this work utilize clap sound to actuate the gesture tracking mode to move the robot and deactivate the gesture tracking mode after last ceasing the robot. For moving the wireless robot the controls are presented using the hand gestures from Kinect sensor, we have used 5 hand gestures (STOP, FORWARD, RIGHT, LEFT and REVERSE). The clap sound signal and control signals to generate different gestures are developed using MATLAB program. These signals are then passed to the robot to move it in the designated directions.