Quantifying judgmental uncertainty: Methodology, experiences, and insights

Probability encoding is the process of extracting and quantifying subjective judgments about uncertain quantities. In the early 1970's the Decision Analysis Group at SRI International recommended a formal process for probability encoding based on a structured interview between a trained interviewer and the subject. Additional methods have been developed for processing encoded probabilities to facilitate their use in decision analysis calculations, including methods for aggregating probability distributions obtained from multiple experts and methods for discretizing continuous probability distributions to simplify computations. Since the original development of these methods, literally hundreds of applications have been conducted as part of corporate-sector decision analyses, public-policy analyses, and decision analysis training seminars. The SRI encoding process and its extensions are reviewed, and some examples and insights regarding the techniques that decision analysts have found effective for quantifying judgmental uncertainty are presented.