Multi-paragraph Reading Comprehension with Token-level Dynamic Reader and Hybrid Verifier

Multi-paragraph reading comprehension requires the model to infer answers of arbitrary user-generated questions by reasoning cross-passage information. Previous work usually generates answer by directly employing a pointer network to predict the start and end position of the answer. However, span-level reading is insufficient since intermediate words may matter more. In this paper, we propose a novel unified network that includes a selector, a Token-level dynamic reader, and a Hybrid verifier (TH-Net). The core of token-level dynamic reader is a gate mechanism which dynamically selects important intermediate words according to boundary words. We decide the reader score from each token being both the boundary and the content. Moreover, we adopt a hybrid network verifier considering semantic answer-answer and entailment question-answer relationships to robust the model in case of being fooled by adversarial answers. Our experiments on SQuAD-document, SQuAD-open, and Trivia-wiki datasets show significant and consistent improvement as compared to other baselines and achieve the state-of-the-art performance on two of them.