ARthroNeRF: Field of View Enhancement of Arthroscopic Surgeries using Augmented Reality and Neural Radiance Fields

Arthroscopy is a minimally invasive orthopedic procedure commonly used to treat joints such as the shoulder, hip, or knee. A major difficulty for surgeons in arthroscopic procedures is the simultaneous coordination of the surgical tools used for manipulation and the arthroscopic camera. To further complicate this task, the narrow space where arthroscopic procedures are performed limits the ability to move the arthroscope inside the patient’s body, restricting the field of view. In this work, to overcome these limitations, we introduce ARthroNeRF, a novel framework that combines Neural Radiance Fields (NeRF) and Augmented Reality (AR). This framework allows for the generation of synthetic viewpoints from the perspective of surgical tools without the need for an additional camera. To evaluate the feasibility of the proposed framework, we conducted a user study with 18 participants. In this study, participants were tasked to touch hidden targets assisted by a synthetic view generated from the perspective of a surgical tool. The results of this study demonstrate that ARthroNeRF provides accurate supplementary visual information and suggest that ARthroNeRF has the potential to streamline the learning process in arthroscopic surgery. In addition, we built a system capable of presenting the reconstructed scenes using the Microsoft HoloLens 2. The incorporation of AR, overlaying synthesized images alongside the original arthroscopic footage and within the surgeon’s visual field, represents a viable alternative to enhance perception in spatially constrained scenarios.