Suturing Support by Human Cooperative Robot Control Using Deep Learning

Considering the widespread use of surgical robots in recent years, computer-assisted surgery is becoming significantly popular. Because the automation of surgical tasks without human intervention remains complex owing to individual patient differences, a human cooperative control is proposed in this study. A system in which an operator manipulates one surgical instrument to insert a suture needle was developed, along with another surgical instrument that automatically pulls out the needle from the operated instrument. In the proposed method, YOLOv3 and a standard convolutional neural network (CNN) are used to estimate the penetration and pull state of the needle. An image-based state estimator classifies the state regardless of the stiffness of the object to which the suture needle is inserted. Furthermore, after the pull state is detected, despite a failure of the needle pulling, the position can be corrected and the automated surgical instrument can approach the needle again. Experiments on human cooperative control demonstrated the effectiveness of state estimation using the proposed method. In addition, the failure of grasping observed in a previous study caused by the needle angle error was reduced.