Comparative Analysis of Transformer based Models for Question Answering

There have been numerous significant developments in Natural language processing (NLP) in past few years. Now language models can learn improved representations of the natural language due to the Transformer and the self-attention mechanism. On numerous NLP benchmarks, these attention-based models have produced excellent, cutting-edge results. The rising popularity of transfer learning is one of the contributing causes. This paper presents a comparison between some of the transformers models like BERT, ALBERT, RoBERTa, XLNET, DistilBERT, Electra, Pegasus etc. and the application of some of these models for Question-Answering (QA) system using SQUAD2 dataset. There are a lot of machine learning algorithms that which helps in solving question answering task using NLP. Previously "Bag of Words" was most common method for answering questions that were presumed by the developers. But this was a very time-consuming process for developers to frame each pair of question answer. This was very useful for chatbots but it was not able to answer question for large database. Nowadays with the help of transformers libraries, HuggingFace developed a BERT Question Answering model which scans a specified document provided by user and tries to answer the questions relating to that document. It can quickly answer from a large document.