Machine vision augmented displays for teleoperation

The authors describe a machine vision-based aid for teleoperation that provides the operator with information not available or readily apparent on the camera views. This aid is intended to allow the operator to perform a task effectively and efficiently where communication bandwidth may be limited. All 6 degrees of freedom of the object's pose can be computed from a single view by a monocular camera, which may be mounted at any known orientation relative to the manipulator tool tip. The pose of the object is then communicated to a graphic display system that creates an overlay on the operator's live video display. The operator can thus teleoperate the robot guided by the computed pose information. No a priori knowledge of the location of the manipulator or target object is required, other than visibility from a camera. An overview of the facilities used to implement this operator aid and apply it to simulations of space servicing operations is provided.<>