Evaluating Fine-Tuned Small Language Models for ICD-10 Diagnosis Coding

Medical coding is crucial for healthcare system's data analysis and resource allocation, but hospitals face staffing challenges in coding. This study evaluates the capabilities of openaccess, Small Language Models (SLMs) for automated ICD-10 diagnosis coding. Using the MIMIC-IV dataset with over 100,000 real clinical notes, we fine-tuned and evaluated three crossdomain and domain-specialized biomedical SLMs ranging from 1 to 7 billion parameters: Falcon-RW-1B, Meditron-7B, and Phi3-Mini-4K-Instructor-3.5B. We implemented parameter-efficient fine-tuning methods, including Low-Rank Adaptation (LoRA) and Quantization, to optimize efficiency. Our results demonstrate that model size does not necessarily correlate with performance, as the 3.5B parameter Phi3-Mini-4K-Instructor-3.5B achieved the highest weighted F1-score of 0.72, outperforming the larger model. Longer context windows proved crucial for accuracy, though training times extending beyond 22 days for larger models raise practical deployment concerns.