Multi-Keyframe Guided Points Tracking Method for Enhanced Robotic Surgery

Points tracking provides target guidance for the basic tasks of robotic surgery, such as autonomous cutting, autonomous suturing, and surgical navigation. Existing invivo tracking algorithms mainly consider region-level tracking, making it difficult to achieve pixel-level accuracy. This paper proposes a multi-keyframe guided tracking method (MKTracking) for robotic surgery, which achieves long-term dynamic tracking of any point on endoscopic images. An incremental keyframe pool building block based on time interval and image appearance is proposed, which contains keyframe basic information, tracking patch and tracking points. Besides, an online multi-keyframe query mechanism is designed to efficiently retrieves the three most similar templates from the keyframe pool based on the number of feature matches within the target patch. This mechanism can realize the identification of target moving out and the re-tracking of target reappearance. Points tracking result is obtained by fusing the optical flow results within the target patch across multiple frames. We evaluated MKTracking using ground-truth data, and the results demonstrated that our approach achieved superior tracking performance.