Paraphrase Identification With Deep Learning: A Review of Datasets and Methods

The rapid advancement of Natural Language Processing (NLP) has greatly improved text-generation tools like ChatGPT and Claude, offering significant utility but also posing risks to media credibility through paraphrased plagiarism—a subtle yet widespread form of content misuse. Despite progress in automated paraphrase detection, inconsistencies in training datasets often limit their effectiveness. This study examines traditional and modern approaches to paraphrase identification, revealing how the under-representation of certain paraphrase types in widely-used datasets, including those for training Large Language Models (LLMs), undermines plagiarism detection accuracy. To address these issues, we introduce and validate ReParaphrased, a refined paraphrase typology, and extend the Extended Typology Paraphrase Corpus (ETPC) with meticulous manual annotations to enhance reliability. Using the augmented ETPC, we fine-tune the LLama3.1-7B-instruct model, uncovering significant disparities in paraphrase type distribution across existing datasets. A detailed analysis of the MRPC benchmark dataset further highlights critical distributional issues and their implications. We propose four key solutions to address dataset limitations, providing both theoretical and practical guidance for improving dataset quality. These contributions aim to establish a more robust foundation for NLP model training and evaluation. Finally, we outline future research directions and suggest improvements for dataset development to advance AI-driven paraphrase detection.