GeroSim: A simulation framework for gesture driven robotic arm control using Intel RealSense

Intel RealSense™ is an exciting new technology from Intel that offers a more immersive and intuitive human-computer interaction (HCI) with hand and head tracking and gesture recognition. Robotics has been an ever evolving area of research and development. Robotics has been extensively used in automation. Several past works have been focused towards developing robots that can respond to human voice, gesture in a more natural ways. However, a simulation framework has never been proposed for gesture driven robotic control. Realistic simulation of robotic arm that can mimic the actual actions of a robot and respond to exactly the same way is essential in designing several applications including automation. This paper provides a unique framework for robotic arm simulation for gesture driven control called GeroSim (Gesture-Robotics-Simulator). The simulation engine is logically bind with Internet of Things (IoT) to enable remote controlling. Any control mechanism can be added as a plugin to the interface to test the interface viability before adopting real hardware. Proposed work presents a unique scripting language called Geroscript that enables different modalities to control a robot seamlessly. Results shows that the engine can support multiple modalities and can be used for practical design of gesture controlled robotic systems.