Multi-model Comparison for Classification of Medical Records using the BioBERT Models

The contents of medical records are extremely crucial for clinical diagnosis and medication, but there is often too much information to accurately find the key message when medical decision making. Therefore, the NLP (Natural Language Processing) and BERT (Bidirectional Encoder Representations from Transformers) models can be used to extract the main characteristics of the medical records from each patient. This study compares the performance of BERT, BioBERT and PubMedBERT for classification of four clinical divisions from medical records. After text preprocessing, modeling and testing, PubMedBERT model shows the best result with an accuracy 0.9, precision 0.81 and the weighted F1-score 0.8. The results of this study verify that PubMedBERT is the best classification model, which can be used to extract patient features to fully understand the key background information of patients, and to improve the medical decision making, and reduce medical disputes and unessential costs.