Entity Recognition in Clinical Text Using A Hybrid Model Based on LSTM and Neural Networks

Successful analysis of clinical text is a challenge with sparse data available on the internet due to various issues. This paper proposes a model for Clinical text analysis to correctly predict and annotate the various keywords or entities present in a medical text which is mostly unstructured. The proposed solution uses deep learning models and NLP techniques. In this model, The basic training of the neural network is done using Bi-directional LSTM and Conditional Random Fields. The model uses an NLP pipeline that has the following stages: document assembler, sentence detector along with tokenization functions, and clinical Named Entity Recognition and Named Entity Recognition converter. With this model, the accuracy obtained was close to 86% on test data, of which individually the Named Entity Recognition converter has an overall accuracy of 90 percent. Improved accuracy was obtained by setting certain hyper-parameters and changing the required deep learning models even with smaller size of data.