A Hybrid Approach Towards Automated Essay Evaluation based on Bert and Feature Engineering

Educational institutions often assess a student's critical thinking and communication skills based on essay responses. Manual Evaluation is time-consuming, and there may be wide variations when multiple evaluators rate batches of essays. In the last few years, the automated grading of essay scripts has emerged as a new area of research. Most studies essentially focus on visible attributes such as length, vocabulary, sentiment or spelling. The use of neural networks requires the conversion of text into some vector representations. However solely using handcrafted attributes or text encodings implies primarily operating on word granularity. On the other hand, Transformers can handle dependencies between words of the text. In this paper, we propose a hybrid model that can capture the interaction of words in the essay using the BERT self-attention transformer, along with handcrafted syntactical features. While previous studies have built individual models for every essay topic, our model has been incrementally trained on multiple essay topics to test its generalizability. The validation of the model uses quadratic weighted kappa to compare human-rated scores and model scores.