Multimodal Teleoperation of Heterogeneous Robots within a Construction Environment

Automation in construction continues to be a topic of interest for many in industry and academia. However, the dynamic environments presented in construction sites prove these tasks to be difficult to automate reliably. This paper proposes a novel method of teleoperation for multiple heterogeneous robots within a construction environment. The system is achieved by creating a virtual reality interface that allows an operator to control multiple robots both synchronously and asynchronously. Feedback is provided from an array of RGBD cameras, force sensors, and precise odometry data. The DRC-Hubo and Spot robot platforms are used for implementation and experimentation. Experiments include useful tasks for construction including item manipulation and item delivery of tools and components. Results demonstrate the feasibility of implementing the system in a construction environment, including trajectory comparisons, task learning curves, and successful multi-robot collaboration.