Preliminary result of vision-based teleoperation of an UAV and UGVs team

This research's final goal is that a user can teleoperate the team of an UAV and multiple UGVs with only vision information under unstructured environment, especially GPS-denied environment. For this scenario, we show the preliminary result of a novel vision-based teleoperation framework of a unmanned aerial vehicle (UAV) and unmanned ground vehicles (UGVs) team. The system consists of a master (a user and a haptic device) and slaves (one UAV and three UGVs) parts. The video show that once the user only teleoperates the UAV, then UGVs follow the UAV autonomously due to the vision-based control. Vision-based control guarantees convergence of image feature to the desired image feature in the our previous work[1]. This control allows robots mainatain coordination. We use backstepping-based control for low-level control of UAV (trajectory tracking). We are extending the result to the team of an UAV and multiple UGVs and designing the suitable haptic feedback device in order to provide intuitive teleoperation framework to the user. Hardwares — UAV: Ascending Technology Hummingbird — UGV: Our own ground robot named TROT which is omni-directional. — Haptic device: Force dimension — Camera: smartphone (Samsung galaxy S2) — UAV position, velocity, rotation information — Vicon motion capture system. Softwares — Image processing: OpenCV