Multi-Document Summarization Using Selective Attention Span and Reinforcement Learning

Abstractive text summarization systems using recently improved RNN-based sequence-to-sequence architecture have shown great promise for single-document summarization. However, such neural models fail to perpetuate the performance in the multi-document summarization setting owing to the long-range dependencies within the documents, overlapping/contradicting facts and extrinsic model hallucinations. These shortcomings augment the model to generate inconsistent, repetitive and non-factual summaries. In this work, we introduce
REISA
, a sequence-to-sequence model with a novel
reinforced selective attention span
that attends over the input and recalibrates the local attention weights to focus on important segments while generating output at each time step.
REISA
utilizes a reinforcement learning-based policy gradient algorithm to reward the model and formulate attention distributions over the encoder input. We further benchmark
REISA
on two widely-used multi-document summarization corpora – Multinews and CQASumm, and observe an improvement of
$+2.91$
and
$+6.64$
ROUGE-L scores, respectively. The qualitative analyses on semantic similarity by BERTScore, faithfulness by question-answer evaluation and human evaluation show significant improvement over the baseline-generated summaries.