Improving Retrieval-Based Question Answering with Deep Inference Models

Question answering is one of the most important and difficult applications at the border of information retrieval and natural language processing, especially when we talk about complex questions which require some form of inference to determine the correct answer. In this paper, we present a two-step method that combines information retrieval techniques optimized for question answering with deep learning models for natural language inference in order to tackle the multiple-choice question answering problem. In the first stage, each question-answer pair is fed into an information retrieval engine to find relevant candidate contexts that serve as the underlying knowledge for the inference models. In the second stage, deep learning architectures are used to predict if a candidate answer can be inferred from the context extracted in the first stage. We deploy multiple deep learning architectures pre-trained on different datasets in order to capture semantic features and to enlarge the scope of the questions we can answer correctly. As it will be described, each dataset used for training the inference models has particular characteristics that can be exploited. In the end, all these solvers are combined in an ensemble model to predict the correct answer. This proposed two-step model outperforms the best retrieval-based solver by over 3% in absolute accuracy. Moreover, the model can answer both simple, factoid questions and more complex questions that require reasoning or inference.