Handling occlusion in Augmented Reality surgical training based instrument tracking

In the medical field, research studies have shown that Augmented Reality (AR) based surgical training has a good potential in making the learning process more active. However, lack of displaying the correct occlusion between the real surgical instrument and virtual organ greatly limits the trainee surgeons understanding and reduces the overall system usability. In this paper, we propose a novel mutual occlusion handling method based on surgical instrument tracking and 3D positioning approach in AR environment. Therefore, we introduce a monocular image processing based paradigm that aims at (1) tracking the instrument using both background subtraction and Hough transform method (2) calculating the 3D position of the instrument using the geometry of perspective projection (3) comparing the 3D coordinates of the real instrument with the virtual organ to achieve a realistic AR rendering system. The experimental results show that our approach is highly accurate and can handle the mutual occlusion automatically in real time.