Clinical Report Classification: Continually Learning from User Feedback

Hospitals have to deal with continually arriving clinical data. In this paper, we present a neural network approach that can aide the process medical document classification. In this scenario a human annotator can correct the model predictions for a new training sample which can then be used for training. This data needs to be classified into ICD categories and the newly obtained knowledge should be captured by the model with minimal loss of already acquired knowledge. More specifically, different strategies are proposed and evaluated for constructing a replay dataset in a continual learning setting. The presented methodology alternates an incremental learning phase with a full retraining of all training samples seen so far. In this manner, a balance can be found where most of the time newly obtained knowledge can immediately be added to the model, but not to the extent where it loses a vital part of previously obtained knowledge.