IslamicPCQA: A Dataset for Persian Multi-hop Complex Question Answering in Islamic Text Resources

Today, one of the most significant challenges faced by Islamic question-answering systems is responding to multi-hop complex questions using various information sources. Multi-hop complex questions, which include bridge and comparative questions, are types of complex queries that require multi-hop reasoning and the use of at least two paragraphs from different documents to answer. On the other hand, in recent years, most research in the field of answering multi-hop complex questions has focused on the English language, and limited effort has been made for low-resource languages, including Persian. To improve the efficiency of questionanswering systems in supporting low-resource languages such as Persian, and to enhance the quality of responses to multi-hop complex questions this paper introduces the Persian dataset “IslamicPCQA.” This dataset consists of 12,282 question-answer pairs and is the first Persian dataset for answering multi-hop complex questions, created exclusively using content from unstructured sources, specifically from nine Islamic encyclopedias. To create this dataset, a unique framework was developed, including stages of data collection, preprocessing, constructing a content recommendation graph infrastructure or hyperlink-based graph, and annotating the data, which can be utilized for creating similar datasets in other low-resource languages for the task of answering multi-hop complex questions. To evaluate and benchmark this dataset, some of the best pre-trained language models that support Persian were fine-tuned using this dataset, and the results were reported based on two metrics: F1 score and Exact Match. The findings indicate that this dataset significantly enhances Persian question-answering systems, with the XLM-RoBERTa model achieving an F1 score of 80.44% and an Exact Match of 67.33%, outperforming other baseline models.