Extraction of patient subpopulations using transformer-encoded vectors

We demonstrate a novel pipeline for extracting and classifying complex subsets of documents from a broader set. We demonstrate this method using a Kaggle classification task to categorize patient visit notes. This method first uses a named-entity recognition (NER) tool, here a clinical metathesaurus, to select terms of interest from reports. These terms are vectorized using a transformer model. The vectors' dimensions are reduced using a dimensionality reduction method - either the encoder section of an auto encoder, or UMAP. At this point, the vectors can be clustered for unsupervised learning, or classified using a feed-forward neural network (FFNN) with an attention head if prior categories are known. We demonstrate this method on a freely-available dataset of patient notes, classifying them into one of five specialties (neurology, gastroenterology, orthopedics, radiology, or urology). Our results show that it is possible to separate complex populations with minimal text pre-processing using such a pipeline, and that it is possible to achieve this separation with either labeled examples from the dataset of interest, or with labels taken from an exterior source, such as expert knowledge. Such a system can be valuable for discovering distinct types of documents from a broader set, or for separating them into predefined subsets.