Abstractive Text Summarization Using BERT for Feature Extraction and Seq2Seq Model for Summary Generation

Humans are very easy to access various existing data about text using search engines and text generate applications. Text summarization is a technological development of the Natural Language Processing branch which takes points and main information from a paragraph. This is done to increase efficiency in understanding a paragraph or document. In this study, we used the type of abstractive text summarization, which is a summarization resulting from deeper sentence understanding. That the resulting summarization will produce a new summary that is not always in the input text, so this will increase the reader's understanding. In text summarization, various methods have been used such as Support Vector Machine (SVM), Long Short Term Memory (LSTM), Bayesian Networks, and so on. In this study, we used pre-trained model Bidirectional Encoder Representations (BERT) derived from transformers, and Sequence to Sequence (Seq2Seq) and the dataset used was the CNN Daily Mail dataset. The pre-trained BERT model is used to perform feature extraction, the results will be forwarded by Seq2Seq for summarization. We also conduct evaluations using ROUGE, by comparing the results of the summary model with the results of the summary made by humans. ROUGE's evaluation results show that typing accuracy in shorter articles is not significantly different from that in longer articles.