Learning Autonomous Surgical Irrigation and Suction With the da Vinci Research Kit Using Reinforcement Learning

The irrigation-suction process is a common procedure to rinse and clean up the surgical field in minimally invasive surgery (MIS). In this process, surgeons first irrigate liquid, typically saline, into the surgical scene for rinsing and diluting the contaminant, and then suction the liquid out of the surgical field. While recent advances have shown promising results in the application of reinforcement learning (RL) for automating surgical subtasks, fewer studies have explored the automation of fluid-related tasks. In this work, we explore the automation of both steps in the irrigation-suction procedure and train two vision-based RL agents to complete irrigation and suction autonomously. To achieve this, a platform is developed for creating simulated surgical robot learning environments and for training agents, and two simulated learning environments are built for irrigation and suction with visually plausible fluid rendering capabilities. With techniques such as domain randomization (DR) and imitation learning, two agents are trained in the simulator and transferred to the real world. Individual evaluations of both agents show satisfactory real-world results. With an initial amount of around 5 grams of contaminants, the irrigation agent ultimately achieved an average of 2.21 grams remaining after a manual suction. As a comparison, fully manual operation by a human results in 1.90 grams remaining. The suction agent achieved 2.64 and 2.24 grams of liquid remaining across two trial groups with more than 20 and 30 grams of initial liquid in the container. Fully autonomous irrigation-suction trials reduce the contaminant in the container from around 5 grams to an average of 2.42 grams, although yielding a higher total weight remaining (4.40) due to residual liquid not suctioned. Further information about the project is available at https://tbs-ualberta.github.io/CRESSim/ Note to Practitioners—The irrigation-suction process is a surgical procedure for rinsing and cleaning surgical fields. This work tackles automating the process to reduce the workload for surgeons. Our approach is based on two customized simulation environments that can simulate the irrigation and suction process realistically. Two autonomous agents are trained using robot learning approaches in the environments for completing irrigation and suction, respectively, and then transferred to the real world. The agents autonomously control the surgical robot by interpreting the images captured from an RGB camera and the robot’s current state, and generate joint movements of the robot. This approach has been tested in physical settings, showing promising results in terms of the individual and combined performance of the agents in executing the full irrigation-suction process. Future work is needed to extend the approach to more practical surgical settings, evaluate the performance under diverse conditions, and enhance integration with existing surgical robot platforms.