Smart data augmentation for surgical tool detection on the surgical tray

In recent years, several algorithms were proposed to monitor a surgery through the automatic analysis of endoscope or microscope videos. This paper aims at improving existing solutions for the automated analysis of cataract surgeries, the most common ophthalmic surgery, which are performed under a microscope. Through the analysis of a video recording the surgical tray, it is possible to know which tools are put on or taken from the surgical tray, and therefore which ones are likely being used by the surgeon. Combining these observations with observations from the microscope video should enhance the overall performance of the system. Our contribution is twofold: first, datasets of artificial surgery videos are generated in order to train the convolutional neural networks (CNN) and, second, two classification methods are evaluated to detect the presence of tools in videos. Also, we assess the impact of the manner of building the artificial datasets on the tool recognition performance. By design, the proposed artificial datasets highly reduce the need for fully annotated real datasets and should also produce better performance. Experiments show that one of the proposed classification methods was able to detect most of the targeted tools well.