Real-Time Ultra-Fine-Grained Surgical Instrument Classification

Accurate classification of ultra-fine-grained surgical instruments can significantly reduce the rate of canceled or postponed surgical procedures and improve a hospital's overall operational efficiency. However, accurately classifying these instruments is challenging due to the vast number of surgical instruments in a hospital's Central Sterile Services Department (CSSD) and their ultra-fine-grained distinctions. To address this challenge and assist CSSD technicians, we propose a real-time ultra-fine-grained surgical instrument classification system. Our system consists of a unique open-environment image acquisition platform and multi-view CNN and transformer-based architectures to capture and classify multi-view images of instruments in real-time. We train models on images from three globally recognized surgical trays: Eye Vitrectomy, Major Laparotomy, and Minor Laparotomy, encompassing 95 distinct classes. We evaluate our system in real-time and on image-based datasets, demonstrating state-of-the-art (SoTA) performance. A user study conducted after deployment in a hospital CSSD reveals that the system significantly improves workflow efficiency, streamlining CSSD operations.