Responsible AI for Government Program Evaluation and Performance Audits

With the proliferation of user-friendly generative-AI tools, many government agencies, such as the National Science Foundation and the National Institutes of Health, have started exploring how these tools could improve one of their primary functions: managing grant programs to distribute funds for research. Government bans on the use of generative-AI for initial grant application evaluation limits the use of this technology. However, this research shows that a post hoc evaluative approach within the domain of Responsible AI provides a means by which government agencies can evaluate the big data collected through their grant programs. This proposed framework, Responsible AI for Evaluation (RAI-E), achieves this goal by curating big governmental administrative data using grant evaluation rubrics followed by model fine-tuning for agency-specific evaluation tasks. Convergence between model- and human-generated evaluations then enables robust diagnosis of grant program equity and fairness via Responsible AI and algorithm auditing methods. Preliminary experiments show improved model performance through the instruction-tuning of open source language models for evaluating government grant applications. Further research will explore the auditing of the underlying algorithms, devising a method for algorithm-in-the-loop frameworks to aid government administrators in evaluating their human-driven program performance.