An abdominal phantom with instrument tracking for laparoscopic training

We developed an abdominal phantom with an embedded stereo camera for tracking multiple hand-held instruments inserted through entry ports. This system can be used for training laparoscopic surgeons, as well as for training bedside assistants in robotic surgery. We present the computer vision methods used to track multiple instruments in real time, with a system evaluation that shows frame rates of 26.6 fps for a 672×376 image and 11.6 fps for a 1280×780 image and corresponding latencies of 38 ms and 87 ms, respectively, when tested on a portable PC platform with an 11th Gen Intel CPU running at 2.8 GHz. The mean Euclidean distance error of the instrument tracking is 2.0 mm in the 672p case and 2.8 mm in the 1280p case. Additionally, the tracking information drives virtual instruments in a simulated environment, which generates improved visualizations of the surgical scene, such as a top-down view and/or a “laser” virtual extension of the instrument. We perform a user study with 10 novice subjects to compare these improved visualizations to the baseline case (only endoscope view) and the results indicate that the combined top-down view and laser extension enhancements provide a statistically significant performance improvement. In the future, the simulator could also improve the (visual) realism of the training platform and could be part of a larger system that enables simultaneous training (and skill assessment) of multiple members of a surgical team, such as the surgeon and first assistant in da Vinci robotic surgery.