Path Generation with Reinforcement Learning for Surgical Robot Control

In the field of robotic surgery, Robot-Assisted Minimally Invasive Surgery(RAMIS) has shown its great potential of benefiting both surgeons and patients in the past few decades of research and practice. The current trend of RAMIS targets towards a higher level of autonomy in carrying out surgical tasks. However, most real RAMIS tasks still rely on manual control, thus the performance mostly depends on the dexterity of the surgeon. Their fatigue or small errors could cause life-threatening damages to the patients, especially high-workload surgeons. Since corrections and errors are inevitable in manual control, the actual tool paths in real operations are often deviated from ideal trajectories. For robot Learning from Demonstrations(LfD), these sub-optimal paths would eventually affect the robot's learning performance. Therefore, much research is being explored in enhancing the performance of robot-generated instrument tool paths and at the same time reducing the reliance on manual manipulation demonstrations in surgical robot learning. In this paper, both Reinforcement Learning and Learning from Demonstration are used to generate a smooth moving trajectory without the use of manual robotic control kinematics data. Two tasks, peg transfer and pattern cutting, were chosen to verify the performance. The method was trained and validated in simulations, namely Asynchronous Multi-Body Framework (AMBF) and Moveit. Then da Vinci Research Kit is used to validate the real case performance. The results have shown that this path generation framework could automate given repetitive surgical tasks, and potentially adapted to other surgical tasks.