Evaluating Correct-Consistency and Robustness in Code-Generating LLMs

Ensuring the reliability of large language models (LLMs) for code generation is crucial for their safe integration into software engineering practices, especially in safety-critical domains. Despite advances in LLM tuning, they frequently generate incorrect code, raising concerns about robustness and trustworthiness. This PhD research introduces Turbulence, a novel benchmark and evaluation framework designed to assess both the correct-consistency and robustness of LLMs through a structured coding question neighbourhood approach. By evaluating model performance across sets of semantically related but non-equivalent coding tasks, Turbulence identifies discontinuities in LLM generalisation, revealing patterns of success and failure that standard correctness evaluations often overlook. Applied to 22 instruction-tuned LLMs across Python coding question neighbourhoods, the benchmark highlights significant variability in correctness, including error patterns persisting even under deterministic settings. Future work will extend the question neighbourhood concept to Capture The Flag (CTF) challenges, enabling a deeper analysis of model reasoning capabilities in progressively complex tasks. This extension has attracted interest from the UK AI Safety Institute, which recognises the frame-work's potential for advancing rigorous evaluation methodologies in the context of safe and trusted AI for software engineering.