Probing with Precision: Probing Question Generation for Architectural Information Elicitation

Software Requirements Specifications (SRS) often lack the necessary level of specificity required by software architects to make well-informed architectural decisions. This deficiency compels software architects to probe business analysts to collect more details pertinent to architectural requirements from the clients. In our previous work, we introduced Probing Question-flows (PQ-flows) that can assist business analysts to probe stakeholders and gather architecturally significant information for the creation of a more comprehensive SRS. Key limitations of our previous work were the manually created templatized PQ-flows and the mapping of PQ-flows to the software requirements based on standard Vector Space Model. In this study, we propose a Retrieval Augmented Generation (RAG) prompting framework to address these limitations. We conducted experiments using ChatGPT and Mistral-7B models. We present our findings utilizing human and automated evaluation metrics on a subset of the publicly available PUblic REquirements (PURE) dataset.CCS CONCEPTS• Software and its engineering → Requirements analysis; • Computing methodologies → Natural language generation; • Information systems → Information retrieval query processing; Question answering.