Comparing the Performance of a Large Language Model and RxNorm Mapping for Detecting Medication Dispensing Errors from Real-world Data

This study compares the performance of two systems—RxNorm, a rule-based standardized medication naming system based on National Drug Codes (NDCs), and GPT-4, a large language model (LLM)—for detecting dispensing errors using real-world pharmacy data. Our findings show that RxNorm outperformed GPT-4 in specific categories such as ingredient, strength, and dose form for dispensing error detection, while GPT-4 demonstrated superior performance in the composite category by leveraging its free-text interpretation capabilities. Notably, RxNorm's structured approach was more reliable in cases involving ambiguous or inconsistent free-text medication descriptions, highlighting its advantage when NDCs were available. Despite this, GPT-4's reliance on free-text descriptions enabled it to flag errors that RxNorm missed, particularly when discrepancies arose between medication descriptions and linked NDCs. These results suggest a hybrid model, integrating the complementary strengths of both systems with pharmacist oversight, has the potential to optimize dispensing error detection, reduce false positives, and enhance medication dispensing safety.