Diverse Distractor Generation for Constructing High-Quality Multiple Choice Questions

Distractor generation task aims to generate incorrect options (i.e., distractors) for multiple choice questions from an article.Existing methods for this task often utilize a standard encoder-decoder framework. However, these methods often tend to generate semantically similar distractors, since the same article representations are used to generate different distractors. Multiple generated distractors with similar semantics are considered equivalent. Because the correct answer is unique, students can eliminate these distractors even without reading the article. In this paper, we propose a multi-selector generation network (MSG-Net) that generates distractors with rich semantics based on different sentences in an article. MSG-Net adopts a multi-selector mechanism to select multiple different sentences in an article that are useful to generate diverse distractors. Specifically, a question-aware and answer-aware mechanism are introduced to assist in selecting useful key sentences, where each key sentence is coherent with the question and not equivalent to the answer. MSG-Net can generate diverse distractors based on each selected key sentence with different semantics. Extensive experiments on the RACE dataset and Cosmos QA dataset show that the proposed model outperforms the state-of-the-art models in generating diverse distractors.

I. Introduction

Multiple choice question (MCQ) is the most popular form of assessment for reading comprehension [1], which is widely used in many well-known tests, e.g., GRE, TOEFL, and SAT. As shown in Fig. 1, a typical MCQ contains three elements: (i) question; (ii) correct answer, which is a single gold answer, i.e., the option of (A); (iii) distractors, which are alternative answers used to interfere with the students to select the correct answer, e.g., the options of (B1) and (B2) [1]. Besides, a complete test of MCQ usually contains an article, which provides reading comprehension materials for candidates to answer a question. The quality of an MCQ depends heavily on the quality of its multiple distractors. When distractors cannot interfere with the student selecting the correct answer, an MCQ will become simple and the test will lose the ability of the assessment. However, it needs a lot of time to manually construct high-quality distractors. Therefore, automatically generating high-quality multiple distractors is an important and challenging task [2].