Enhancing ICD-9 Code Prediction with BERT: A Multi-Label Classification Approach Using MIMIC-III Clinical Data

This study addresses the challenge of accurately predicting ICD-9 codes from clinical notes in the MIMIC-III database, focusing on intensive care unit (ICU) records. Existing methods using the MIMIC-III dataset, such as CAML, have achieved significant results using convolutional neural networks. In this paper, we leverage BERT (Bidirectional Encoder Representations from Transformers) to enhance the multi-label classification of ICD-9 codes from clinical text. We pre-process the MIMIC-III dataset to address the sequence length constraints imposed by BERT, employing techniques like automatic truncation and section-based filtering to optimize input data. Our model utilizes the BERT architecture fine-tuned on the top 50 ICD-9 codes to improve prediction accuracy, employing binary cross-entropy for multi-label classification. Experimental results show that our approach achieves competitive performance, with a Micro-F1 score of 0.83 after a single epoch using sequences of length 128. Future work will explore using longer sequence lengths and compare performance with existing CAML preprocessing methods. This study demonstrates the potential of BERT in advancing automated ICD-9 code assignment in clinical settings, providing a robust framework for future improvements in healthcare informatics.