PhyNeSS: A Physics-driven Neural Networks-based Surgery Simulation system with force feedback

In this work we present PhyNeSS - a novel physics-driven neural networks-based surgical simulation system which, for the first time, combines the complexity and accuracy of physics-based non-linear soft tissue models and commercial finite element codes with the high speed of execution of machine learned neural networks. While soft tissue is inherently nonlinear, physics-based simulation of nonlinear tissue behavior with haptic feedback is very challenging as the solution of the coupled nonlinear partial differential equations is iterative and therefore extremely computationally intensive. The major contribution of this paper is that through an unprecedented combination of hard and soft computing methods, it is able to reduce the solution of nonlinear problems to almost the same complexity as solving linear problems. This promises to resolve one of the longest-standing technical challenges of real time surgical simulation. The first phase of the method is a pre-computation phase, in which each node of the organ model, with known linear or nonlinear material properties, is provided with carefully chosen prescribed displacements and the response, computed using commercial finite element software tools, is recorded off-line and stored in a large database. The data in then vastly condensed into a set of coefficients describing neurons of a radial basis function (RBF) networks or easier storage and rapid reproduction. During real-time computations, as the surgical tool interacts with the organ models, these neural networks are used to reconstruct the deformation fields as well as the reaction forces at the surgical tool tip. We present simulation results for realistic surgical scenarios with real time force feedback.