Spatial Generation of Molecules with Transformers

Generative models play an important role in the discovery of new molecules. However, these models often operate on a string or graph based representation of molecules, without taking spatial information into consideration. In this paper we introduce an autoregressive generative neural network operating directly in the three-dimensional space of atoms. By leveraging the powerful multi-head self-attention mechanism of the transformer architecture the model predicts a set of atoms and a corresponding distance matrix to generate molecules in a translation and rotation invariant manner. We demonstrate that the model can be utilized to generate completely new molecules that resemble the training data in their structural properties. Furthermore, the model is capable of completing unconnected sets of atoms to valid molecular structures.