DeBERTaSSL: Enhancing Molecular Property Prediction with Self-Supervised Learning

This research introduces a novel approach to molecular property prediction, leveraging the transformative power of the DeBERTa model, a state-of-the-art transformer-based architecture. By integrating DeBERTa with graph convolutional neural networks (GCNs) and self-supervised learning techniques using fusion modality, our methodology achieves comprehensive representation learning from SMILES-encoded molecular structures. We combine graph-based and sequential representations to capture both local and global relationships within molecules, while incorporating self-supervised learning enables efficient learning from limited labeled data. Through rigorous evaluation and comparison with existing methodologies, including Graph self-supervised learning, Graph Convolutional Network and others, our model DeBERTaSSL, demonstrates superior performance in predicting molecular properties across diverse datasets. This research contributes to advancing the field of molecular property prediction, offering a novel framework that combines the strengths of transformer-based models with graph-based representations and self-supervised learning techniques.