Towards hybrid control of a flexible curvilinear surgical robot with visual/haptic guidance

Comprised of multiple telescoptic precurved tubes that can independently rotate and translate, concentric tube robots (CTRs) are favorable in minimally invasive surgeries thanks to their small size and considerable dexterity along with curvilinear accessibility. However, there is a lack of investigation on improvement of the surgeons' perception which in turn can be used to guide the telemanipulation. In this work, we proposed an eye-in-hand configuration for the concentric tube robot by adding an endoscope to the tip of the inner tube, which provides direct and intuitive visual sensing ability for the operator. Based on this visual feedback, we further developed two frameworks for the hybrid control of CTR, namely Teleoperation Before Visual Servoing (TBVS) and Teleoperation During Visual Servoing (TDVS). The structures of these two frameworks were elaborated with key algorithms derived. The effectiveness of the proposed methods were demonstrated through a series of experiments both in free space and in a confined environment (inside a skull model). The results manifested that the visual guidance had the potential of assisting the operator to control the CTR more efficiently.