Attribution Scores of BERT-Based SQL-Query Automatic Grading for Explainability

Automated grading of SQL queries poses a challenge due to the complexity of the SQL language and the numerous ways in which one can achieve the same results. While recent advancements in machine learning-based automated grading systems have demonstrated remarkable accuracy, there remains a critical need for providing students with meaningful explanations for the grades assigned by these machines. To address this need, our study focuses on a specific recent work in automated grading for SQL queries. We have developed a comprehensive workflow that leverages machine learning models trained in that work. Our objective is to gain insights into the behavior of the machine learning model, particularly in terms of how it assigns grades to SQL queries. Through our analysis, we have observed that our workflow performs effectively, producing valuable attribution scores for important tokens within SQL queries. These attribution scores shed light on which query components are considered significant by the machine learning model, thus enhancing our understanding of the grading process and facilitating more meaningful feedback for students.