Robotic control with partial visual information

We consider a class of control tasks, which rely from partial visual information in a robotic setting. These tasks are hard in the sense that at every given moment, the available information is insufficient for the control task. Still, the amount of information collected throughout the control process is large and thus seems sufficient for carrying out the task. Such situations commonly arise when the object is frequently occluded from one of the cameras in a stereo pair or when only one moving camera is available. We propose a generic control rule for such tasks and characterize the conditions required for the success of the task. The analysis is based on the observation that mathematically, the behavior of such systems is related to a class of row-action (and POCS) optimization algorithms. In the second part of the paper we focus on one particular task from this class: position and orientation control with a single rotating camera. We show that this task may be carried out, in principle, for any camera movement, and suggest efficient control strategy and camera moving strategy. Interestingly, it seems that the advisable control law is not consistent with simple-minded intuition. We substantiate our claims by simulations and experiments on real data.