DNA-DIR: 2D-3D Geometry Embedding for Intraoperative Partial-to-Full Registration

Intraoperative registration has a broad impact in robotic-assisted surgery (RAS) as it is desirable to have augmented resection guidance by superimposing the preoperative organ mesh on the endoscopic images in RAS. But such non-rigid registration is challenged by the significant partial observation on anatomies due to a limited view frustum. The highly nonstructural content and complex texture in endoscopic images further complicate the challenge. Current investigation suffers from either the strict prerequisite on the initial alignment or the inaccurate annotated datasets. In this work, we first contribute a simulated partial-to-full 2D-3D registration dataset for robotic endoscopic liver surgery with 150 accurate annotated cases from patients with different liver diseases. Additionally, we propose a new progressive registration pipeline, named DNA-DIR, for addressing the partial-to-full 2D-3D non-rigid alignment for RAS with robust pose requirements. Experimental results confirm the effectiveness of the proposed method, facilitating the augmented reality-based assistance closer to the clinical environment.