Learning-based action planning for real-time robot telecontrol with binocular vision in enhanced reality environment

Action planning is one of the pivot issues in robot telecontrol, in which the action instructions are often given by the controller from remote site with the help of vision systems. In this paper, we present a learning-based strategy for action planning in robot telecontrol, in which the parameters of sophisticated actions of the remote robot equipped with a binocular vision system could be pre-scheduled with a virtual robot at the control terminal. The remote robot will then be 'taught' with the scheduled action plan with a series of parameter sets obtained form try-outs with the virtual robot and object in the enhanced environment, thus implementing dedicated actions assigned correctly. The action planning process is implemented within a enhanced reality environment, in which both the virtual and the real robot will be displayed simultaneously for the purpose of being deeply immersed. Experiment results demonstrate that the proposed method is capable of promoting the action precision of the remote robot, and effective and valid to designated applications, where action precision plays a critical role.