Sensory-Motor Manifold Structure Induced by Task Outcome: Experiments with Robonaut

Experiments were performed with Robonaut to determine if repeated teleoperation through a task could delineate a manifold in the robot's sensory-motor state-space (SMSS) that was separable by the outcome of the task. A teleoperator guided the robot through 13 trials of a pick and place task. By design, in 5 of the trials were successful and 8 were not. A time series of instantaneous sensory-motor state vectors was recorded at 8Hz from the robot throughout the trials. Support vector machine (SVM) analysis suggested that the individual vectors were separable with a 70% probability if both sensory and motor information were used. The analysis revealed, however, little structural information beyond the promise of reasonable separability. Therefore, dimensionality reduction techniques were applied. These embed high-dimensional vectors into a lower dimensional space where patterns may be discernible. Singular value decomposition, multidimensional scaling, and spatio-temporal isomap (STI) analysis of the time series revealed a 3D structure within the SMSS that was dependent on, and separable by task outcome. An STI embedding generated by 2 successful and 4 unsuccessful trials was seen to be sufficient for the projection and classification of the remaining trials. A comparison of that result with the SVM classification demonstrated that STI outperformed SVM. It is conjectured that STI's reliance on the temporal evolution of the time-series gave it the advantage over the SVM analysis which was time independent. It was also found that Robonaut's haptic sensory data degraded the outcome separability if used without processing. However, lateral inhibition of the haptic signals significantly enhanced separability in both the SVM and STI analyses