LLaVA-Assisted Prompt Engineering for Liver X-Ray Report Analysis

Integrating artificial intelligence (AI) with healthcare revolutionizes diagnosis and patient care by improving clinical documentation, managing electronic health records, and interpreting medical images, ultimately improving accuracy and efficiency. In regions like Egypt, which faced the world’s highest mortality rate from liver disease and the second-highest deaths from liver cancer in 2022, advanced AI tools such as the multimodal Large Language and Vision Assistant (LLaVA) have significant potential to address global challenges in liver disease diagnosis. LLaVA has emerged as a critical tool to support the challenges in disease diagnosis. LLaVA can classify and describe patient scan images, extract information from clinical notes, and provide accurate diagnoses through effective prompt engineering. Additionally, it facilitates answering questions, summarizing data, and translating complex medical terminology into simple language, enabling patients to better understand their medical reports and diagnoses. This paper emphasizes LLaVA’s role in medical applications, focusing on prompt engineering to enhance the effectiveness of pre-trained models. Experiments using LLaVA demonstrated promising results for liver tumor classification with quantitative metrics such as accuracy, precision, and recall that exceeded 90%. These findings align with extensive research on medical image and text analysis, highlighting the potential of LLaVA architecture to assist healthcare professionals in diagnosing complex cases.