Developing COVID-19 Information validation system Using Natural language inference

In the recent COVID-19 pandemic outbreak, a lot of information about the disease spread without being able to validate its truth. In this work, we contribute to develop a COVID-19 information validation system, the truth of which refers to scientific articles. The system consists of two main modules, namely the fact-finding module and the sentence comparison module. The fact-finding module looks for relevant facts to the validated information, while the sentence comparison module leverages Natural Language Inference (NLI) task by using deep learning to compare validated information and facts relevant to that information. This work focuses on building the NLI model, with RoBERTa-Large trained on SNLI and MultiNLI dataset. The model was evaluated on SNLI dev and test set and Multi NLI dev set (matched and mismatched), with median accuracy of 0.931, 0.925, 0.9031, and 0.9042, respectively. The model was also evaluated on stress test for NLI with an accuracy of 0.7466 on the competence test, 0.8776 on the noise test, and 0.6884 on the distraction test. We also created our own NLI dataset for further evaluation, which is created based on rumors and circulating information about COVID-19 and facts from the CORD-19 dataset, containing 100 sentence pairs of premise and hypothesis. Model evaluation on this dataset resulting in an accuracy of 0.84, which means the model already works well on sentences from the CORD-19 dataset.