Multi-Task Learning for Lung Disease Classification and Report Generation via Prior Graph Structure and Contrastive Learning

Analyzing medical records takes a large amount of time and resources for medical workers. Therefore, the advancement of report generation technology is critical for saving doctors’ time while producing reports. Some diseases frequently exhibit co-occurrence interactions in the context of medical picture categorization and report creation, which necessitate hu-man-designed and added representation. To further capture this latent information, we propose creating a disease co-occurrence matrix from the existing dataset and training a graph neural network on it. We use multi-label contrastive learning to assist the model in distinguishing the mutual links between different diseases. Meanwhile, we incorporate report retrieval module and decoder model to complete the report generation task, and the multi-task learning of disease classification and report generation can improve the generalization ability of the target task. Experimental results show that our pro-posed combined method and multi-task learning approach have shown significant improvement compared to previous research.