Improving Text Summarization with Transformer Networks and Reinforcement Learning for English

As the volume of text data continues to grow rapidly, efficient and accurate text summarization has become crucial for managing and processing this information. Traditional summarization methods, including extractive and abstractive approaches, face challenges in maintaining relevance, fluency, and factual consistency, particularly in complex domains such as legal documents. In this research, we propose an advanced summarization framework leveraging transformer networks, specifically designed to enhance the quality of English text summaries. The framework begins with the preprocessing of text data, including lowercasing, elimination of punctuation, HTML tags, stopwords, and stemming/lemmatization to standardize the input. Text feature extraction is then performed using the Term Frequency-Inverse Document Frequency (TF-IDF) method, converting textual data into numerical representations that highlight the significance of terms within documents. The core of the framework employs a transformer model that eschews traditional RNNs in favor of multiple attention layers and positional encoding to capture global dependencies and enable parallel processing. The model's encoder-decoder structure uses self-attention mechanisms and multi-head attention layers to effectively encode the sequence and context of words, leading to high-quality abstractive summaries. The proposed architecture's effectiveness is evaluated using the BillSum dataset, which includes over 22,000 U.S. Congressional bills and human-written summaries. The results highlight the superior performance of the proposed Transformer model, which achieved the highest ROUGE-1, ROUGE-2, and ROUGE-L scores, with values of 37.03, 16.89, and 28.34, respectively and the Proposed method is implemented using python software. Our approach demonstrates significant improvements in summary accuracy and relevance compared to existing methods, making it a promising solution for automated summarization tasks across various domains.