Identifying Alcohol-Related Information From Unstructured Bilingual Clinical Notes With Multilingual Transformers

As a key modifiable risk factor, alcohol consumption is clinically crucial information that allows medical professionals to further understand their patients’ medical conditions and suggest appropriate lifestyle modifying interventions. However, identifying alcohol-related information from unstructured free-text clinical notes is often challenging. Not only are the formats of the notes inconsistent, but they also include a massive amount of non-alcohol-related information. Furthermore, for medical institutions outside of English-speaking countries, these clinical notes contain both a mixture of English and local languages, inducing additional difficulty in the extraction. Thanks to the increasing availability of electronic medical record (EMR), several previous works explored the idea of using natural language processing (NLP) to train machine learning models that automatically identify alcohol-related information from unstructured clinical notes. However, all these previous works are limited to English clinical notes, thereby able to leverage various large-scale external ontologies during the text preprocessing. Furthermore, they rely on simple NLP techniques such as the bag-of-words models that suffer from high dimensionality and out-of-vocabulary issues. Addressing these issues, we adopt fine-tuning multilingual transformers. By leveraging their linguistically rich contextual information learned during their pre-training, we are able to extract alcohol-related information from unstructured clinical notes without preprocessing the clinical notes on any external ontologies. Furthermore, our work is the first to explore the use of transformers in bilingual clinical notes to extract alcohol-related information. Even with minimal text preprocessing, we achieve extraction accuracy of 84.70% in terms of macro F-1 score.