Will you Trust me More Than Chatgpt? Evaluating LLM-Generated Code Feedback for Mock Technical Interviews

Technical interviews are an opportunity for candidates to showcase their technical proficiency to employers. Despite social aspects of tech interviews, preparation for these assessments is often done in isolation. Cooperative preparation methods, such as mock interviews—or simulated interviews between peers—can provide useful feedback and reveal candidates' technical deficiencies. However, this process is rarely used by candidates in practice. Further, current technical interviews lack authentic feedback for candidates on technical evaluation criteria, such as code correctness, optimality, and complexity. To resolve this, we designed a website to facilitate mock technical interviews and provide users with LLM-generated feedback on code using ChatGPT. We devised a between-subjects study with 46 participants across two settings: human-administered (a live human sharing generated feedback) and automated (direct LLMgenerated feedback). Our results show that candidates perceive coding feedback as useful—however, we found participants regard automated feedback as less trustworthy compared to feedback provided by humans. In light of our findings, we discuss implications and guidelines for incorporating trustworthy AI in technical interview feedback systems.