An Analysis of the Cost of Validating Semantic Composability

Validation of semantic composability is a non-trivial problem and a key step in component-based modeling and simulation. Recent work in semantic composability validation promise to reduce verification, validation, and accreditation efforts. However, the underlying cost of current validation approaches can undermine the promised benefits, and the trade-off between validation accuracy and validation cost is not well understood. In this paper we present, to the best of our knowledge, the first quantitative study on the cost of validating semantic composability. Firstly, validation approaches are categorized into techniques that validate general model properties, and techniques that validate model execution. Secondly, we focus on two key factors that influence validation cost, namely, simulation problem characteristics and the validation approach adopted. Our study covers four representative validation approaches, CD++ DEVS, Z-based DEVS, Petty and Weisel formal validation, and deny-validity, and for simplicity, we use computation time as a measure of validation cost. Based on a queueing network model with 1,000 components, the cost of validating general model properties accounts for 55% of total validation time, with the remaining 45% incurred by model execution validation. With respect to composition structure, a 10% increase in the fork and join component interconnections increases validation cost by more than half. In model execution validation, the time-based deny-validity approach is seven times more expensive than the timeless Petty and Weisel formalism.