The implementation of augmented reality in a robotic teleoperation system

The Wheelchair Mounted Robotic Manipulators (WMRM) are prevalently used to help the elderly or people with spine injury to conduct Activities of Daily Living (ADL). In this project, we extended its working range by making such system teleoperatable. In addition, we enhanced its efficiency by providing more natural and intuitive method for manipulation. With Augmented Reality (AR) technology, our system could present the reconstructed 3D scene of remote area at local station in an interactive way. Besides adjusting perspective and scale of the display, the gesture operating on virtual object would also be converted into control commands of robotic arm in real-time. Series of experiments have been carried out to demonstrate the function of our system.