Transformer-Based Refinement of Pseudoknot Predictions in RNA Secondary Structures

Accurate prediction of RNA secondary structures including pseudoknots is crucial for understanding RNA bio-logical functions. However, traditional methods for predicting RNA secondary structures often struggle to capture long-range base-pairing interactions involved in pseudoknots. We propose a novel hybrid framework that accepts noisy predictions of RNA secondary structures and refines them with a transformer-based deep learning model. To make this possible, we first introduce a novel encoding scheme that seamlessly fuses RNA primary sequence and secondary structure information into linear sequences in an expanded alphabet of 28 symbols. In this new language setting, a novel tokenization scheme is further introduced to enable the transformer model to learn and leverage the constraints governing stable RNA folding from the training data. By training on a dataset of experimentally validated RNA structures, we show that our model can correct erroneous predictions, adding missing pseudoknots and removing spurious ones. Furthermore, we compared our approach with the trRosetta tool, obtaining satisfactory results that demonstrate the effectiveness of our method.