A practical and universal framework for generating publicly available medical notes of authentic quality via the power of crowds

Medical notes written by doctors in hospitals or clinics are information-rich. However, in many countries or cultures, few people have access to them for educational and research purposes, even once anonymized. This is because their contents, including patients’ disease information, are sensitive and require confidentiality. Therefore, publicly available pseudo-medical notes are needed. Authentic pseudo-medical notes must meet two requirements: (1) medical consistency, and (2) informal descriptions and specific sub-language; however, these are empirical knowledge, even for medical doctors, and are not clarified specifically. We combat this by harnessing the power of crowds. We propose a human-in-the-loop framework for generating publicly available professional medical notes utilizing human cognitive traits with a small dataset. The practical and universal framework has three steps. In Step 1, crowd workers imitated actual notes. In Step 2, crowds and algorithms collaboratively identified notes’ characteristics based on comparisons between actual and dummy notes. In Step 3, the texts generated in Step 1 that exhibited the characteristics from Step 2 were evaluated as authentic medical notes that met all requirements. We demonstrated this framework with a total of 1,662 crowds’ power. All data were preprocessed to protect patients’ privacy before the experiments. The crowds’ generated 9,756 notes were evaluated as the most realistic compared to dummy medical notes written by doctors. These crowd-generated medical notes, which are the largest publicly available dataset of Japanese medical notes, are published. This study was the first challenge for the crowds to solve the medical expert-level task.