An Autonomous Dynamic Camera Method for Effective Remote Teleoperation

In this paper, we present a method that improves the ability of remote users to teleoperate a manipulation robot arm by continuously providing them with an effective viewpoint using a second camera-in-hand robot arm. The user controls the manipulation robot using any teleoperation interface, and the camera-in-hand robot automatically servos to provide a view of the remote environment that is estimated to best support effective manipulations. Our method avoids occlusions with the manipulation arm to improve visibility, provides context and detailed views of the environment by varying the camera-target distance, utilizes motion prediction to cover the space of the user’s next manipulation actions, and actively corrects views to avoid disorienting the user as the camera moves. Through two user studies, we show that our method improves teleoperation performance over alternative methods of providing visual support for teleoperation. We discuss the implications of our findings for real-world teleoperation and for future research.