Interventionalist Hand Motion Recognition With Convolutional Neural Network in Robot-Assisted Coronary Interventions

The development of procedural competencies by interventional cardiology fellows for minimally invasive treatment of cardiac diseases depends on their effective training and evaluation. Learning tool manipulation for safe robotic PCIs requires expert supervision and use of high-fidelity systems, however, with limited proficiency for real-time hand motion recognition. Therefore, this study proposes a deep learning-based model for identifying operators’ actions. The proposed model is based on the convolutional neural network (CNN) that consists of 1-D convolutional layers for automatic feature map extraction, downsampling, and fully connected layers (FCLs) for inference. The developed models were evaluated using a multimodal dataset collected from sensory glove, electromagnetic (EM), and surface electromyography (sEMG) sensors and real-time angiograms in the course of in vivo catheterization trials performed by nine subjects (two experts and seven novices) using a custom-built robotic catheter system (RCS). The results indicate that the model achieves a 92%–96% accuracy in identifying five actions across four clusters compared with a recognition performance of 83% when recognizing all six actions. Furthermore, we compared the proposed model performance with the existing studies, and the analyses show that our model has a 2%–3% higher accuracy for five-action recognition. Therefore, the proposed model could be employed for real-time hand motion recognition in robot-assisted percutaneous coronary intervention (R-PCI) trials.