Eye Gaze Map as an Efficient State Encoder for Underwater Task Automation

ROVs are the primary tool used in marine environments for conducting dexterous manipulation tasks and the teleoperation of which requires experienced operators. In an attempt to ease the fatigue of the operators and to potentially improve task efficiency, we propose an image based framework to automate underwater routine tasks via imitation learning. In particular, we utilize the gaze information of the operator for extracting task-relevant information from the raw image input by an encoding network. We further demonstrate that the incoporation of the eye gaze encoder facilitates the training of the task policy which includes both maneuver and decision making.