TCNAEC: Advancing Sentence-Level Revision Evaluation Through Diverse Non-Native Academic English Insights

In the domain of Natural Language Processing (NLP), the English Writing Fluency Improvement for non-native speakers, particularly in academic contexts, poses significant challenges. While Sentence-level Revision (SentRev) endeavors to address this concern, the existing evaluation corpus, SMITH, falls short in offering a robust and comprehensive assessment of the task. To bridge this gap, our research offers a novel evaluation corpus generation scheme, leading to the creation of Ten-Country Non-native Academic English Corpus (TCNAEC). A meticulous analysis revealed the superior characteristics of TCNAEC over SMITH in various dimensions. Our evaluation also uncovered intriguing linguistic phenomena, offering valuable insights for fellow researchers. In contrast, the Grammatical Error Correction (GEC) task, which shares similarities with SentRev, has been more extensively explored, resulting in a richer set of training and evaluation corpora. However, the distinctive attributes of SentRev present a heightened challenge in NLP implementation. The TCNAEC, representing ten countries, captures the unique English expression styles of non-native speakers worldwide, offering a more holistic view compared to the Japan-centric SMITH. Furthermore, while SMITH primarily revolves around computational linguistics, TCNAEC spans multiple disciplines, accentuating its comprehensiveness. The construction strategy of TCNAEC, ensuring semantic consistency between Draft and Reference, emphasizes meaningful structural variations, reflecting the stylistic disparities between non-academic and academic texts.