Object Detection and Tracking in Ultrasound Scans Using an Optical Flow and Semantic Segmentation Framework Based on Convolutional Neural Networks

Based on non-ionizing radiation, ultrasound scanning is safe to image a specific region of the body repeatedly to identify and localize target anatomical structures during therapeutic and diagnostic procedures. However, it is labor intensive, and requires sonographers to have extensive experience to be able to identify and track these anatomical structures of interest, making the identification and tracking process highly prone to errors. In this paper, we propose a framework to autonomously detect, localize and track anatomical structures in ultrasound scans during scanning and therapeutic sessions in real-time. The proposed framework uses a segmentation-based convolutional neural network (CNN) to detect and localize the target anatomical structure within a scan. Concurrently, it uses an optical flow CNN to track the movement of this structure across frames to accurately guide therapeutic procedures. We tested the framework on detecting and tracking the Vagus nerve in ultrasound scans. It achieved state-of-the art localization and tracking accuracy with an average error of less than 1.25 mm for localization and 0.75 mm for tracking while maintaining an inference time of less than 35 ms.