This paper addresses the issue of investigating the relationship between skin potential response (SPR) and robot motion from the aspect of auditory and visual modalities. The motion of the robotic devices primarily gave stimulus to the human through the auditory and visual modalities. Our previous work indicated that actuation noise caused from the robots tended to raise the sympathetic nerve system response of the heart rate variability. However, it was difficult to evaluate instantaneous response corresponding to a specific stimulus using the HRV. The SPR is possible to reflect the instantaneous response to the stimulus. In the experiment, we controlled the conditions of the visual and/or auditory modalities by blocking out the sound and sight from the robot, a manipulator with 5 D.O.F. We also changed the position of the robot around subjects for stimuli variety. As the result, it is shown that the SPR magnitudes, under the condition of blocking out not only the sound but also the sight from the robot, tended to be significantly lower than without blocking.