Teaching LLMs Non-Functional Requirements Modeling: A Grammar and RAG Approach

A picture is worth a thousand words. Non-Functional Requirements (NFRs), such as security and usability, are modeled using Softgoal Interdependency Graphs (SIGs) to capture potential conflicts and synergies. However, the practice of NFR modeling remains limited, partly due to unfamiliarity with modeling languages like SIG and insufficient understanding of relevant NFRs. Large Language Models (LLMs) show some knowledge of NFRs and SIG concepts, such as goal decomposition and operationalization, but often lack precise knowledge of formal SIG syntax. We introduce SIG-GPT, a GPT-4-based LLM augmented with SIG knowledge using text-based grammar supplied and Retrieval Augmented Generation (RAG). RAG enhancs LLM responses by retrieving relevant external knowledge, while the grammar enforces correct syntax, guiding the LLM to generate SIGs align with formal notation. To help practitioners better understand SIG modeling, reduce time and effort, and enhance NFR proficiency, we apply textual grammar to SIG-GPT, ensuring it is ready for seamless integration with visual modeling tools like RE- Tool, enabling the LLM to generate correct SIG structures without requiring a large dataset of SIG examples. Results show that SIG-GPT with grammar and RAG achieves 100% syntactic accuracy, 95% semantic accuracy, and 98% cohesion (CCR) while aligning with Bloom's Taxonomy to enhance structured reasoning in SIG modeling.