Structuring unstructured clinical narratives in OpenMRS with medical concept extraction

We have developed a new software module for the open source Electronic Medical Record System OpenMRS to analyze unstructured clinical narratives. This module leverages Named Entity Recognition (NER) to deliver concise, semantic-type driven, interactive summaries of clinical notes. To this end, we performed an extensive empirical evaluation of four Named Entity Recognition (NER) systems using textual clinical narratives and full biomedical journal articles. The four NER systems under evaluation are MetaMap, cTAKES, BANNER. We studied several ensemble approaches built upon the above four NER systems to exploit their collaborative strengths. Evaluations are performed on the manually annotated patient discharge summaries from the Informatics for Integrating Biology and the Bedside group (I2B2) and the CRAFT dataset. The main results include (1) BANNER significantly outperforms the other three systems on the I2B2 dataset with F1 values in the range of .73-.89, in contrast to .28 - .60 of other systems; and (2) Surprisingly, an ensemble approach of BANNER with any combinations of the other three approaches tends to degrade the performance by .08 - .11 in F1 when evaluated on the I2B2 dataset. Based on the evaluation results, we have developed a BANNER-based NER module for OpenMRS to recognize semantic concepts including problems, tests, and treatments. This module works with OpenMRS versions 2.×. The user interface presents concise clinical notes summaries and allows the user to filter, search and view the context of the concepts. We have also developed a companion web application to retrain the BANNER model using data from OpenMRS. The module and source code are available at wiki.openmrs.org/display/docs/Visit+Notes+Analysis+Module.