Developing a Tuberculosis Q&A Database Using Q&A-ET: Question and Answer Evaluator Toolkit

Large language models (LLMs) have shown remarkable potential in various natural language processing tasks, including text generation, question-answering, etc. However, their application in specialized domains like medical research remains limited due to their tendency to produce harmful or inaccurate responses. This is especially true in the field of microbiology, here on Mycobacterium tuberculosis (MTB), where generating reliable and safe medical advice is critical. Creating domain-specific question-answer datasets essential for fine-tuning LLMs is labor-intensive and time-consuming. In this study, we present Q&A-ET (Question and Answer Evaluator Toolkit), a framework designed to streamline the generation of high-quality question-answer datasets by leveraging both human feedback and the capabilities of LLMs. Our approach not only reduces the time and effort required to build datasets but also enhances the accuracy and reliability of LLMs in the medical domain. We will open-source both the toolkit and the MTB question-answer dataset, which consist of two articles with 55 unique questions and 124 expert-evaluated question-answer pairs reviewed by a medical expert, offering valuable resources for future research in this critical area.