Vision-Based Markerless Tracking for Continuum Surgical Instruments in Robot-Assisted Minimally Invasive Surgery

Robotic-assisted single-port minimally invasive surgery (MIS) can reduce trauma and shorten hospitalization. However, surgical instruments for robotic single-port surgery usually possess multiple joints without distal joint sensors. Hence, closed-loop control is not directly applied to these joints, which may result in lowered motion accuracy. Aiming at enabling markerless closed-loop control without introducing additional sensing hardware, this letter proposes a vision-based markerless tracking method for continuum single-port surgical instruments. Firstly, a color-enhanced random forests (RFs) segmentation method is proposed, using a color-based method for annotating endoscopic images. Secondly, based on the segmented surgical end effector and the level-set framework, an optimization is formulated for 3D pose estimation, utilizing the kinematics of the continuum surgical instrument. The experimental results show that the proposed vision-based marker-less tracking method is effective for tracking continuum surgical instruments.