Evaluating Large Language Models for Criminal Profiling: A Framework-Based Approach

We investigate whether off-the-shelf large language models (LLMs) can produce reliable criminal profiles without task-specific fine-tuning. To this end we propose Profiling-LLM, a prompt-based framework that operationalizes four established criminological models—Narrative Action System, Sexual Behavioural Analysis, Sexual Homicide Pathways, and Spatial Behavioural Analysis—to structure LLM outputs. Twelve state-of-the-art LLMs are evaluated on six landmark homicide cases. We measure (i) reasoning completeness, (ii) framework-label agreement, and (iii) intra-framework consistency. Agreement varies by model and framework, but even the best systems deliver coherent, fully populated profiles in more than 90% of prompts. The results suggest that general-purpose LLMs, when steered by criminological theory, can assist early-stage profiling, while highlighting where specialized fine-tuning or human oversight remains essential.