Enhancing Large Vision Language Models for Liver CT Scans in Medical Reports

This study leverages multimodal Large Vision-Language Models (LVLMs), such as LLaVA, IDEFICS, and Flamingo, to enhance liver disease diagnosis in high-burden regions like Egypt, which reports the world's highest liver disease mortality rate and second-highest liver cancer mortality (WHO, 2022). By analyzing the Eurorad's medical imaging dataset and extracting insights from clinical notes, LVLMs achieve over 90 % accuracy, precision, and recall in liver tumor classification. Additionally, LVLMs translate complex medical terminology into plain language, improving patient comprehension. Experiments demonstrate LVLMs' efficacy in radiology applications, positioning them as viable decision-support tools for hepatology in resource-constrained settings. Index Terms-Large Vision-Language Models (LVLMs), Large Language Model (LLM), Large Language and Vision Assistant for BioMedicine (LLaVA-Med), Contrastive Language-Image Pretraining (CLIP), Eurorad Dataset