Transforming Sketches of UML Use Case Diagrams to Models

Considering that use case specification often plays the most important role in requirements engineering, transitioning from a design on paper to an editable, standard-compliant model is a fitting candidate for automation. This paper presents a methodology for transforming Use Case diagram sketches into standardized UML Use Case models. The methodology utilizes a tailored Faster R-CNN network for detecting objects in the sketch, combined with the Google Cloud Vision API for text recognition. It also includes techniques for detecting and identifying diagram relations, their direction, start, and end points, and integrating these elements into a structured UML diagram. The final output is transformed into an XML Metadata Interchange (XMI) format file, ensuring compatibility with various UML Computer-Aided Software Engineering (CASE) tools. The methodology was implemented in a tool and empirically validated through experiments using a dataset of 32 Use Case diagram sketches, made up of 765 instances of Use Case diagram elements. The tool’s performance demonstrates 94% accuracy in symbol recognition and overall diagram digitization accuracy of 95%.