Development of a Visual Guidance System for Laparoscopic Surgical Palpation using Computer Vision

Currently, there are numerous obstacles to performing palpation during laparoscopic surgery. The laparoscopic interface does not allow access into a patient's body anything other than the tools that are inserted through small incisions. Palpation is a useful technique for augmenting surgical decision-making during laparoscopic surgery, especially when discerning operations involving cancerous tumors. In this study, a visual guidance system is proposed for use during laparoscopic palpation, specifically engineered to be part of a motion-based laparoscopic palpation technique. In particular, the YOLACT++ model is used to localize a target organ, the gall bladder, on a custom dataset of laparoscopic cholecystectomy. Our experiments showed an AP score of 90.10 for bounding boxes and 87.20 on masks. In terms of the speed performance, the model achieved a playback speed of approximately 20 fps, which translates to approximately 48 ms video latency. The palpation path guides are guidelines that are computer-generated within the identified organ, and show potential in helping the surgeon implement the palpation more accurately. Overall, this study demonstrates the potential of deep learning-based real-time image processing models to complete our motion-based laparoscopic palpation system, and to realize the promising role of artificial intelligence in surgical decision-making.