The main aim behind the design of rehabilitation robotic systems is to support disabled people in daily life situations as well as in the working environment. This requires the autonomous execution of different tasks. In order to achieve the ability of the robotic system to operate autonomously, the sensor system for the scene observation providing the necessary inputs for the manipulator control is essential. In this paper, the main emphasis is on the development and integration of the closed-loop controls at the visual sensory input level in order to increase the robustness of the vision-based system control. Moreover, the integration of additional sensors in order to support the vision system and to increase the reliability of the whole robotic system is discussed.