Liberate Radiologists from Tedious Work: Large Language Model Enhanced Contrastive Learning for Ultrasound Report Generation

Automatic report generation has the potential to alleviate the heavy workload of radiologists. However, existing methods often generate low-quality reports. To this end, we propose a large language model enhanced contrastive learning (LLM-ECL) approach to improve the quality and accuracy of breast ultrasound report generation. LLM-ECL leverages the pretrained MiniGPT-4 to extract visual features and employs an encoder-decoder architecture to generate reports. To address the issue that models trained with conventional cross-entropy loss tend to generate reports lacking specific abnormality descriptions, we introduce the contrastive loss. We perform contrastive learning between visual features and textual features generated by LLaMA2 to enhance image-text consistency. Experiments conducted on a breast ultrasound dataset comprising 655 patient cases demonstrate that LLM-ECL produces reports with high professionalism and accuracy.