Transferring Virtual Surgical Skills to Reality: AI Agents Mastering Surgical Decision-Making in Vascular Interventional Robotics

Vascular interventional surgery offers advantages, such as minimal invasiveness, quick recovery, and low side-effects. Performing automatic guidewire navigation on vascular surgical robots can effectively assist doctors in performing surgery. Deep learning and reinforcement learning methods have been widely used for guidewire navigation tasks. However, the challenge remains in making delivery decisions for complex and extended pathways, with real-time images being the only data source. The development of network architecture, coupled with the formulation of an efficacious training regimen for this network is of significant importance and holds substantial meaning for the advancement of autonomous systems in vascular surgical robots. Therefore, this research proposes a virtual training environment that incorporates real vascular projections to create virtual environment. In this environment, the approach is enhanced by incorporating guidewire tip-to-target distance in the reward function, using real-time images as input states. This article also employs a multiprocess proximal policy optimization algorithm to accelerate training process and a multistage training approach to reduce the training difficulty. Results demonstrate the effectiveness in virtual automated guidewire navigation and improves success rates. This research proposes a method, which generates effective inputs for the reinforcement learning agent, and enables the pretrained agent to accomplish delivery tasks in real-world scenarios.