Y-Net: A Deep Convolutional Architecture for 3D Estimation of Contact Forces in Intracardiac Catheters

Estimating the applied forces in three dimensions during catheter-based surgeries can make the entire process tangible for surgeons. This will result in diminishing the risk of fatal errors while improving the outcome of the surgery. In this work, a novel deep convolutional neural network is proposed to estimate the applied forces to the tip of the catheter in x, y and z direction directly from the images. As a sensor-free method, this end-to-end architecture extracts the features of the catheter's deflections through a stereo vision system. The network is fed by a two-channel image at a time. The images are captured from the top and side views through a designed mechanical setup. This is a simulation of the biplane fluoroscopy in which the target point (e.g., catheter) is visualized from two different orientations. Given the images of the catheter from two points of views, the proposed network is able to fuse the images of both angles and translate the deflections to the corresponding forces in 3D. The evaluation results revealed that the proposed system properly fuses the input images and maps them to the force space as a regressor. The output of the system was assessed and the average of five times of system runs showed the mean-absolute error of
$< \text{0.0040}\;N$
. To the best of the authors’ knowledge, currently the reported results are the state-of-the art in the literature.