A Reliable Kinematic Measurement of Upper Limb Exoskeleton for VR Therapy with Visual-inertial Sensors

Virtual reality (VR) is a powerful technology that provides a structured and safe environment for ADL training, allowing patients to have a similar experience as real-world training. However, a limited robot sensing system is proven reliable for such training. The effectiveness of current robot sensors was limited due to inherent technical problems, such as the installation challenges of encoders and IMU’s drifting, acceleration, and magnetic issues. Thus, we propose a novel and reliable sensing system consisting of absolute rotary encoders and visual-inertial sensors for the upper-limb exoskeleton in VR therapy. Our sensing system has demonstrated angle measurement for various robot joint types, including hinge, ball, and revolute joints along the limb’s longitudinal axis. Its sensing feedback can construct virtual arms that interact with virtual objects in the VR environment. In our experiments, with Vicon as the ground truth, our visual-inertial sensors achieved root-mean-square errors smaller than 2.3491° and a strong correlation (r ≥ 0. 9640, p < 0. 001). Additionally, the experiment result of seven healthy subjects indicated that subjects had similar muscle activations, joint ROM, and joint trajectories in the VR task with our sensing system, compared with the real-world task. Thus, our proposed sensing system can potentially be used in the upper-limb exoskeleton for VR therapy.