Transformer-based Architecture for Assisting Title and Abstract Screening of a Systematic Review

Systematic literature reviews are crucial for decision-making across various fields. Typically, these reviews are carried out using laborious manual methods that rely on, e.g. keyword searches, title, and text analysis. Their outcome is a thematic classification of papers. To alleviate the burden on scientists, several natural language processing-based approaches have been developed to facilitate the screening of papers.Our approach employs pre-trained transformer models, such as Bidirectional Encoder Representations from Transformers (BERT), LITMC-BERT and PubMedBERT, for papers about artificial intelligence research on COVID-19 for the purposes of detection, diagnosis, and prediction. This subject has been growing rapidly over the past three years, thereby making it challenging to be up-to-date with the latest information pertaining to the virus. Furthermore, with the COVID-19 situation appearing to have become more stable, scientists are focusing on investigating the underlying factors that caused the outbreak of the contagion.Our models find relevant documents and categorize them according to their information on COVID-19, such as diseases, treatments, and vaccinations. The proposed models have been evaluated for screening performance using a test set that comprises articles retrieved from the searches of three independent reviewers, who have assessed the eligibility of each paper. Our approach has achieved promising results for researchers and demonstrated its ability to be generalized.