Real-Time 6DOF Pose Estimation of Endoscopic Instruments Using Printable Markers

To accurately navigate surgical instruments around in vivo environments in minimally invasive surgeries through flexible endoscopes or rigid scopes, real-time tracking of surgical instruments is required. It is even challenging to track regular-shaped rigid bodies on a less-distorted general-purpose monocular vision, while most of the surgical instruments or devices used are cylindrical and slim imaged with endoscope distortion in nature. Thus, more dedicated and accurate rigid-body tracking approach for cylindrical endoscopic instruments will be more helpful in intra-operative guidance. In this paper, we present an endoscopic instrument tracking approach with printable markers that consists of a green band and a square with four white and one black circle inscribed in it. The proposed approach can estimate the six degrees of freedom pose, i.e., X, Y, Z, Roll, Pitch, and Yaw, of the imaged instrument with respect to the endoscope. Detailed experiments justify that our proposed marker detection framework allows translation motions between 42 and 100 mm along depth direction and rotational motions around X and Y up to ±180º, with 90% detection rate in dynamic condition, 99% in static condition, and accurate pose estimation with a mean translation error of 1.286 mm with a standard deviation of 0.673 mm and a mean rotational error of 1.497° with a standard deviation of 0.873°. Our proposed marker can be easily printed and wrapped to the existing endoscopic instruments, which potentially can assist in the precise guidance in minimally invasive surgeries without changing the existing workflow or incurring significant cost burdens.