Determining evaluation criteria for a "seal of approval" process for probabilistic risk assessment analysis tools

This paper outlines initial work performed in the design of an approval process to meet the objective expectations, and initial work performed to define appropriate evidence to use in the process. The approval process in general to provide the framework for the discussion of the primary focus of the paper, the presentation of work performed to design and populate a proposed decision support tool, the fitness for use worksheet was first discussed. The worksheet documents an expert assessment of an analysis tool's fitness for use in a selected domain based on the acceptability and importance of presented evidence. A survey eliciting expert opinion on evidence typically used in similar approval processes as well as evidence that should be used in an effort to define evidence to be used in the proposed approval process was conducted. Expert opinion as to the relative importance of evidence was also elicited. The gathered information is used in the initial population of an "evidence database" that supports the worksheet. The wide variety of survey responses as to the importance and use of different evidence reflects, for example, the heterogeneity of the represented communities. Discussing a potential development plan for the approval process concludes the paper.