From Macro to Micro: Autonomous Multiscale Image Fusion for Robotic Surgery

In this paper an automated scanning framework for pCLE and OCT optical biopsy using the da Vinci surgical robot was presented. It is capable of generating large-area mosaics of both pCLE and OCT images. A crucial feature is that pCLE images are used to close the control loop, and mosaicing results from both static and deforming phantoms demonstrated that this effectively compensates kinematic errors. Furthermore, by using OCT images to maintain a constant distance to the tissue, and hence ensure consistent contact between the pCLE probe and the tissue, the system is able to compensate for target motion along the axial direction. This visual servoing allows for the correction of errors due to tissue deformation, robot positioning, and grasping of the pickup probe. The accuracy of this correction is better than the FoV of the pCLE probe, resulting in continuous 2-D mosaics without gaps or discontinuities, which represent a common problem for open-loop control.