Plenoptic cameras in surgical robotics: Calibration, registration, and evaluation

Three-dimensional sensing of changing surgical scenes would improve the function of surgical robots. This paper explores the requirements and utility of a new type of depth sensor, the plenoptic camera, for surgical robots. We present a metric calibration procedure for the plenoptic camera and the registration of its coordinate frame to the robot (hand-eye calibration). We also demonstrate the utility in robotic needle insertion and application of sutures in phantoms. The metric calibration accuracy is reported as 1.14 ± 0.80 mm for the plenoptic camera and 1.57 ± 0.90 mm for hand-eye calibration. The accuracy of needle insertion task is 1.79 ± 0.35 mm for the entire robotic system. Additionally, the accuracy of suture placement with the presented system is reported at 1.80 ± 0.43 mm. Finally, we report consistent suture spacing with only 0.11 mm standard deviation between inter-suture distances. The measured accuracy of less than 2 mm with consistent suture spacing is a promising result to provide repeatable leak-free suturing with a robotic tool and a plenoptic depth imager.