TAJ: A Platform for Integrating Tumor Detection and Depth Perception for Endoscopic Surgery

Surgery for lung cancer requires precision and efficiency, with computer-aided systems displaying signs for enhancing these areas. Yet, the availability of software-based tools to assist in such surgeries remains limited. This paper proposes a collection of software-based tools that will address this issue. This collection includes tumor-detection, a custom depth-estimation algorithm, and a custom user-interface (TAJ) that stores and displays information in real time. Tumors would be detected using YOLOv5, an object detection model, with a confidence baseline of 0.75. This baseline was estab-lished to maximize precision while maintaining a satisfactory recall-which the test results show in a 95.5 % precision and 85.3 % recall. To estimate relative depth, two main approaches were utilized. The first was relative depth estimation using a popular collection of monocular depth algorithm-MiDaS. The three algorithms within MiDaS that were used were “MiDaS-Small,” “DPT-Hybrid,” and “DPT-Large.” The second approach relied on using the detected tumor's frame size in comparison to the dimensions of the camera's feed, with the results being normalized using a modified sigmoid function. This custom algorithm showed a positive trend between real depth and relative depth throughout the domain of the test, whereas the tests with MiDaS did not exhibit as strong of a correlation. Additionally, the custom algorithm was 1.48, 2.97, and 4.33 times quicker than the MiDaS-Small, DPT-Hybrid, and DPT-Large associated algorithms, respectively. The custom user interface was built using Python's Tkinter library, with the live graph being generated using Matplotlib. Thus, this research offers a data-driven solution to enhancing surgery by providing an easy-to-use user interface that offers depth and detection information to surgeons.