An interactive interface for multi-pilot ROV intervention

As we endeavor to deploy and maintain evermore complex infrastructure underwater, the tools used for subsea maintenance and intervention must also evolve to meet new needs. Whether for scientific sampling and exploration, subsea mining, offshore oil & gas production, or installation and maintenance of marine renewable energy devices, there is a regular and increasing need for intervention to conduct maintenance work or repair failed equipment. As work is conducted in hostile, deep and remote locations where divers cannot be used, the use of remotely operated vehicles (ROVs) is commonplace. Despite advances in sensing and machinery, however, significant challenges persist with ROV operations. Inadequate situational awareness, data deluge, and cumbersome operator controls combine to create a technological hurdle that is yet to be overcome. In this work we address the problem of using multiple pilots to collaboratively operate a single robotic system to perform a variety of tasks underwater. This is done via an interactive interface making use of 3D visualization, enhanced feedback using haptics, and an intuitive control scheme that incorporates input from multiple sources. This paper presents a method for fusing multimodal sensor data to build a model-based representation of the ROV and its workspace, details of a prototype ROV manipulator test bed, and results from recent in-water testing where multiple pilots work together to successfully complete a complex underwater task.