Combining Metrics From Clinical Simulators and Sensorimotor Tasks Can Reveal the Training Background of Surgeons

Background: Skill assessment in surgery traditionally has relied on the expert observation and qualitative scoring. Our novel study design demonstrates how analysis of performance in sensorimotor tasks and bench-top surgical simulators can provide inferences about the technical proficiency as well as the training history of surgeons. Methods: We examined metrics for basic sensorimotor tasks in a virtual reality interface as well as motion metrics in clinical scenario simulations. As indicators of the training level, we considered survey responses from surgery residents, including the number of postgraduation years (PGY, four levels), research years (RY, three levels), and clinical years (CY, three levels). Next, we performed a linear discriminant analysis with cross-validation (90% training, 10% testing) to relate the training levels to the selected metrics. Results: Using combined metrics from all stations, we found greater than chance predictions for each survey category, with an overall accuracy of 43.4 ± 2.9% for identifying the level for post-graduate years, 79.1 ± 1.0% accuracy for research training years, and 64.2 ± 1.0% for clinical training years. Our main finding was that combining metrics from all stations resulted in more accurate predictions than using only sensorimotor or clinical scenario tasks. In addition, we found that metrics related to the ability to cope with changes in the task environment were the most important predictors of training level. Conclusions: These results suggest that each simulator-type provided crucial information for evaluating surgical proficiency. The methods developed in this paper could improve evaluations of a surgeon's clinical proficiency as well as training potential in terms of basic sensorimotor ability.