Research on Automatic Text Summarization Using Transformer and Pointer-Generator Networks

Due to the explosive growth of digital content, automated text summary generation has become a critical task for digesting large amounts of information efficiently. In this work, we propose a new method of automatic text summarization based on the combination of Transformer and Pointer-Generator network. The model learns with the existing architecture, leverages the strong context modeling ability of Transformer to capture long-distance dependency, and brings in pointer mechanism in Pointer-Generator network to deal with non-lexical words effectively and enhance the abstraction of summary generation. Firstly, we come up with a multi-layer hybrid framework which integrates the attention mechanism of Transformer with the overlay mechanism of Pointer-Generator to produce a more precise and fluent summaries. This weighted combination can then be refined by implementing a fine-tuning stage that updates the weights of the two networks conditional to the attributes of the input data. Experimental results demonstrate that on the standard dataset, our model outperforms the current best summary generation models with respect to ROUGE score.