Enhancing Automatic Evaluation Systems for Descriptive Answers Using BERT: Addressing Context and Coherence Challenges

This study explores the application of advanced Natural Language Processing (NLP) techniques, specifically leveraging BERT-based models, for evaluating semantic similarity between text inputs. The methodology involves preprocessing text through tokenization, stemming, and sentence chunking, followed by the generation of sentence embeddings using the ‘bert-base-nli-mean-tokens’ model. The model, fine-tuned on Natural Language Inference (NLI) tasks, captures deep contextual relationships, enhancing the ability to compute text similarity through cosine similarity of high-dimensional embeddings. The evaluation used datasets from Semantic Textual Similarity (STS) benchmarks, combined with educational text samples. Key metrics such as Pearson correlation, Spearman correlation, and Root Mean Squared Error (RMSE) were employed to assess the model's performance in predicting similarity scores between text pairs. Results demonstrate that the model achieves a Pearson correlation of 0.53, indicating moderate agreement with human-annotated scores, while the Spearman correlation score of 0.26 highlights a weaker monotonic relationship. An RMSE of 0.12 further suggests low prediction error, supporting the model's efficacy in identifying similarities between text pairs. These findings underscore the potential of BERT-based models in applications like automated essay grading, paraphrase detection, and content-based recommendation systems, where accurate assessment of text similarity is critical