Gesture based robotic arm control for meal time care using a wearable sensory jacket

This work presents the development of a wireless, low cost, wearable sensor jacket for the purpose of controlling a robot arm by mimicking the motion and behaviour of a humans arm. The intended use of our system is to provide remote daily nursing care by using our system from a distant place such as a nursing home or hospital, to control a stationed robotic arm placed in an elderly or patient's home. The final system is comprised of a wearable jacket which is embedded with IMU and flex sensors to detect and track the wearers arm movements and behaviour. The system is capable of detecting up to 5 degrees of freedom of the human arm and replicate these motions using a 6 DOF robot arm. The usability, accuracy and precision of our jacket system is evaluated through a user study and the results demonstrated that our system was more accurate and easier to use for operators than a conventional robotic arm joystick controller. In a water bottle transfer task our developed wearable jacket system demonstrated an average error distance of 29.36mm from the target point, while the results using the conventional joystick demonstrated an average error distance of 37.48mm. Furthermore subjects using our system were able to complete the transfer task in an average time of 44.1s per trial which was more efficient than the joystick method in which subjects averaged 55.55s per trial. Finally, we report a feasibility study with the jacket and a subject to demonstrate the capability of this system of giving a patient water to drink. The feasibility experiment showed an 86.66% success rate in giving a patient water via video stream teleoperation control.