Virtual structure reconstruction and energy estimation of a peptide from a physical tangible interface

Physical models for biological molecules have been used since the 50s to represent their structures at a macroscopic scale. Nowadays, advanced modular non-rigid models (such as the Peppytide tangible interface) successfully approximate the bonding rotational energies and enable realistic folding of small proteins. Such tangible interfaces are also an interesting way to interact with 3D virtual objects but methods to make the 3D physical interfaces converse with virtual models of molecules are still lacking. We present a markerless tracking method to recover the 3D molecular structure of the Peppytide tangible interface from a simple smartphone video, allowing to reconstruct 3D secondary and tertiary structure. This method is based on an up-to-date blob detection to extract reliable 2D positions of atoms from the video and apply a Structure-from-Motion recovery method to get an approximation of the molecule 3D structure. We then estimate the peptide energy after several minimization steps, using classical simulation tools such as GROMACS coupled to a coarse-grained forcefield. The visualisation is done with UnityMol, a molecular editor with advanced rendering features based on the Unity professional game engine, making it easy to add information like the energy of peptide folding.