Generative AI for OCL Constraint Generation: Dataset Collection and LLM Fine-tuning

The Object Constraint Language (OCL) is a formal specification language in model-based systems and software engineering. It defines complex rules and constraints for model-based system design and verification. Constructing an OCL constraint requires expertise not only in OCL syntax but also in meta-model information, which can hinder its application in the practical industrial scenario despite its broad usage. Recently, generative artificial intelligence has demonstrated remarkable performance in code and text generation. This work discusses the generation of OCL constraints from natural language specifications using large language models (LLMs). Given that the automotive and aviation industries are major consumers of model-based engineering, the use of commercial LLMs raises concerns about data privacy. Therefore, we propose to employ open-source and locally deployed LLMs for OCL generation tasks. In this work, we collected a set of meta-models and OCL constraints, which were syntactically validated to ensure the quality of the OCL dataset. Synthetic natural language specifications were generated and used in the dataset for model fine-tuning. Additionally, we designed a retrieval-augmented approach to incorporate meta-model information during LLM fine-tuning and OCL generation. The proposed fine-tuning and OCL generation approach has been experimented with the state-of-the-art open-source LLM, Llama 3 8B. The locally fine-tuned and deployed language model achieved comparable syntactic accuracy and a higher semantic similarity score for OCL generation compared to the cutting-edge commercial models, GPT-4 Turbo and Gemini 1.5 Pro. The usability of the fine-tuned model has been demonstrated for OCL generation in the context of automotive resource allocation.